The multivariate nonparametric Behrens–Fisher problem

A novel longitudinal rank-sum test for multiple primary endpoints in clinical trials: Applications to neurodegenerative disorders

Neurodegenerative disorders such as Alzheimer's disease (AD) present a significant global health challenge, characterized by cognitive decline, functional impairment, and other debilitating effects. Current AD clinical trials often assess multiple longitudinal primary endpoints to comprehensively evaluate treatment efficacy. Traditional methods, however, may fail to capture global treatment effects, require larger sample sizes due to multiplicity adjustments, and may not fully exploit multivariate longitudinal data. To address these limitations, we introduce the Longitudinal Rank Sum Test (LRST), a novel nonparametric rank-based omnibus test statistic. The LRST enables a comprehensive assessment of treatment efficacy across multiple endpoints and time points without multiplicity adjustments, effectively controlling Type I error while enhancing statistical power. It offers flexibility against various data distributions encountered in AD research and maximizes the utilization of longitudinal data. Extensive simulations and real-data applications demonstrate the LRST's performance, underscoring its potential as a valuable tool in AD clinical trials. Nonparametrics, Global test, rank-sum-type test, U-Statistics.

The sex effect on balance control while standing on vestibular-demanding tasks with/without vestibular simulations: implication for sensorimotor training for future space missions

Background: Anatomical differences between sexes in the vestibular system have been reported. It has also been demonstrated that there is a sex difference in balance control while standing on vestibular-demanding tasks. In 2024, NASA expects to send the first female to the Moon. Therefore, to extend the current knowledge, this study attempted to examine whether different sexes respond differently to vestibular-disrupted and vestibular-demanding environments. Method: A total of fifteen males and fifteen females participated in this study. The vestibular function was quantified through different SOT conditions (SOT1: baseline; SOT5: vestibular demanding by standing with blindfolded and sway reference surface). The vestibular stimulation (VS) was applied either unilaterally or bilaterally to vestibular system to induce the sensory-conflicted and challenging tasks. Thus, a total of 6 conditions (2 SOT conditions X 3 VSs: no-VS, unilateral VS, and bilateral VS) were randomly given to these participants. Three approaches can be quantified the balance control: 1) the performance ratio (PR) of center of gravity trajectories (CoG), 2) the sample entropy measure (SampEn) of CoG, and 3) the total traveling distance of CoG. A mixed three-way repeated ANOVA measure was used to determine the interaction among the sex effect, the effect of SOT, and the effect of VS on balance control. Results: A significant sex effect on balance control was found in the PR of CoG in the anterior-posterior (AP) direction (p = 0.026) and in the SampEn of CoG in both AP and medial-lateral (ML) directions (p = 0.025, p < 0.001, respectively). Also, a significant interaction among the sex effect, the effect of SOT, and the effect of VS on balance control was observed in PR of CoG in the ML direction (p < 0.001), SampEn of CoG in the AP and ML directions (p = 0.002, p < 0.001, respectively), and a traveling distance in AP direction (p = 0.041). Conclusion: The findings in the present study clearly revealed the necessity to take sex effect into consideration while standing in vestibular-perturbed or/and vestibular demanding tasks. Also, the results in the present study could be a fundamental reference for future sensorimotor training.

New Confidence Intervals for Relative Risk of Two Correlated Proportions

Biomedical studies, such as clinical trials, often require the comparison of measurements from two correlated tests in which each unit of observation is associated with a binary outcome of interest via relative risk. The associated confidence interval is crucial because it provides an appreciation of the spectrum of possible values, allowing for a more robust interpretation of relative risk. Of the available confidence interval methods for relative risk, the asymptotic score interval is the most widely recommended for practical use. We propose a modified score interval for relative risk and we also extend an existing nonparametric U-statistic-based confidence interval to relative risk. In addition, we theoretically prove that the original asymptotic score interval is equivalent to the constrained maximum likelihood-based interval proposed by Nam and Blackwelder. Two clinically relevant oncology trials are used to demonstrate the real-world performance of our methods. The finite sample properties of the new approaches, the current standard of practice, and other alternatives are studied via extensive simulation studies. We show that, as the strength of correlation increases, when the sample size is not too large the new score-based intervals outperform the existing intervals in terms of coverage probability. Moreover, our results indicate that the new nonparametric interval provides the coverage that most consistently meets or exceeds the nominal coverage probability.

Generalized Nonparametric Composite Tests for High-Dimensional Data

In this paper, composite high-dimensional nonparametric tests for two samples are proposed, by using component-wise Wilcoxon–Mann–Whitney-type statistics. No distributional assumption, moment condition, or parametric model is required for the development of the tests and the theoretical results. Two approaches are employed, for estimating the asymptotic variance of the composite statistic, leading to two tests. In both cases, banding of the covariance matrix to estimate variance of the test statistic is involved. An adaptive algorithm, for selecting the banding window width, is proposed. Numerical studies are provided, to show the favorable performance of the new tests in finite samples and under varying degrees of dependence.

Sample size determination for comparing accuracies between two diagnostic tests under a paired design

With the progressive technology, many medical researches are aimed to develop diagnostic tests that can detect diseases faster and accurately. The assessment of the accuracy of the diagnostic test for classifying two groups is through the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC). When a paired design is considered, the sample size determination requires the information about two AUC estimates and the corresponding variance and covariance of two AUC estimators. This paper derives the nonparametric estimators of the variance and covariance of two AUC estimators. The result is used to derive the sample size formula when the paired sample is planned. Since most of the results do not have a closed form, numerical results are provided under various scenarios.

Directional-sum test for nonparametric Behrens-Fisher problem with applications to the dietary intervention trial

For a nonparametric Behrens-Fisher problem, a directional-sum test is proposed based on division-combination strategy. A one-layer wild bootstrap procedure is given to calculate its statistical significance. We conduct simulation studies with data generated from lognormal, t and Laplace distributions to show that the proposed test can control the type I error rates properly and is more powerful than the existing rank-sum and maximum-type tests under most of the considered scenarios. Applications to the dietary intervention trial further show the performance of the proposed test.

Test Preparation in Figural Matrices Tests: Focus on the Difficult Rules

It is well documented that training the rules employed in figural matrices tests enhances test performance. Previous studies only compare experimental conditions in which all or no rules were trained and therefore ignore the particular influence of knowledge about the easy and difficult rules. With the current study, we wanted to provide some first insights into this topic. Respondents were assigned to four groups that received training for no rules, only the easy rules, only the difficult rules, or for all rules. The results show that a training only for the difficult rules was more effective than the other trainings. This applies also to performance in the easy rules that were actually not part of the training. A possible explanation for this finding is a facilitation of the solution process that is primarily driven by knowledge about the difficult rules. In conclusion, our results demonstrate that taking differences between the rules into account may provide a deeper understanding of the effects of trainings for figural matrices tests.

Bivariate Analogs of the Wilcoxon–Mann–Whitney Test and the Patel–Hoel Method for Interactions

A fundamental way of characterizing how two independent compares compare is in terms of the probability that a randomly sampled observation from the first group is less than a randomly sampled observation from the second group. The paper suggests a bivariate analog and investigates methods for computing confidence intervals. An interaction for a two-by-two design is investigated as well.

Simultaneous inference for factorial multireader diagnostic trials

We study inference methods for the analysis of multireader diagnostic trials. In these studies, data are usually collected in terms of a factorial design involving the factors Modality and Reader. Furthermore, repeated measures appear in a natural way since the same patient is observed under different modalities by several readers and the repeated measures may have a quite involved dependency structure. The hypotheses are formulated in terms of the areas under the ROC curves. Currently, only global testing procedures exist for the analysis of such data. We derive rank-based multiple contrast test procedures and simultaneous confidence intervals which take the correlation between the test statistics into account. The procedures allow for testing arbitrary multiple hypotheses. Extensive simulation studies show that the new approaches control the nominal type 1 error rate very satisfactorily. A real data set illustrates the application of the proposed methods.

A novel nonparametric confidence interval for differences of proportions for correlated binary data

Various confidence interval estimators have been developed for differences in proportions resulted from correlated binary data. However, the width of the mostly recommended Tango’s score confidence interval tends to be wide, and the computing burden of exact methods recommended for small-sample data is intensive. The recently proposed rank-based nonparametric method by treating proportion as special areas under receiver operating characteristic provided a new way to construct the confidence interval for proportion difference on paired data, while the complex computation limits its application in practice. In this article, we develop a new nonparametric method utilizing the U-statistics approach for comparing two or more correlated areas under receiver operating characteristics. The new confidence interval has a simple analytic form with a new estimate of the degrees of freedom of n − 1. It demonstrates good coverage properties and has shorter confidence interval widths than that of Tango. This new confidence interval with the new estimate of degrees of freedom also leads to coverage probabilities that are an improvement on the rank-based nonparametric confidence interval. Comparing with the approximate exact unconditional method, the nonparametric confidence interval demonstrates good coverage properties even in small samples, and yet they are very easy to implement computationally. This nonparametric procedure is evaluated using simulation studies and illustrated with three real examples. The simplified nonparametric confidence interval is an appealing choice in practice for its ease of use and good performance.

Simultaneous statistical inference for epigenetic data

Epigenetic research leads to complex data structures. Since parametric model assumptions for the distribution of epigenetic data are hard to verify we introduce in the present work a nonparametric statistical framework for two-group comparisons. Furthermore, epigenetic analyses are often performed at various genetic loci simultaneously. Hence, in order to be able to draw valid conclusions for specific loci, an appropriate multiple testing correction is necessary. Finally, with technologies available for the simultaneous assessment of many interrelated biological parameters (such as gene arrays), statistical approaches also need to deal with a possibly unknown dependency structure in the data. Our statistical approach to the nonparametric comparison of two samples with independent multivariate observables is based on recently developed multivariate multiple permutation tests. We adapt their theory in order to cope with families of hypotheses regarding relative effects. Our results indicate that the multivariate multiple permutation test keeps the pre-assigned type I error level for the global null hypothesis. In combination with the closure principle, the family-wise error rate for the simultaneous test of the corresponding locus/parameter-specific null hypotheses can be controlled. In applications we demonstrate that group differences in epigenetic data can be detected reliably with our methodology.

A wild bootstrap approach for the selection of biomarkers in early diagnostic trials

BACKGROUND

In early diagnostic trials, particularly in biomarker studies, the aim is often to select diagnostic tests among several methods. In case of metric, discrete, or even ordered categorical data, the area under the receiver operating characteristic (ROC) curve (denoted by AUC) is an appropriate overall accuracy measure for the selection, because the AUC is independent of cut-off points.

METHODS

For selection of biomarkers the individual AUC's are compared with a pre-defined threshold. To keep the overall coverage probability or the multiple type-I error rate, simultaneous confidence intervals and multiple contrast tests are considered. We propose a purely nonparametric approach for the estimation of the AUC's with the corresponding confidence intervals and statistical tests. This approach uses the correlation among the statistics to account for multiplicity. For small sample sizes, a Wild-Bootstrap approach is presented. It is shown that the corresponding intervals and tests are asymptotically exact.

RESULTS

Extensive simulation studies indicate that the derived Wild-Bootstrap approach keeps and exploits the nominal type-I error at best, even for high accuracies and in case of small samples sizes. The strength of the correlation, the type of covariance structure, a skewed distribution, and also a moderate imbalanced case-control ratio do not have any impact on the behavior of the approach. A real data set illustrates the application of the proposed methods.

CONCLUSION

We recommend the new Wild Bootstrap approach for the selection of biomarkers in early diagnostic trials, especially for high accuracies and small samples sizes.

Formulating appropriate statistical hypotheses for treatment comparison in clinical trial design and analysis

We discuss the problem of properly defining treatment superiority through the specification of hypotheses in clinical trials. The need to precisely define the notion of superiority in a one-sided hypothesis test problem has been well recognized by many authors. Ideally designed null and alternative hypotheses should correspond to a partition of all possible scenarios of underlying true probability models P={P(ω):ω∈Ω} such that the alternative hypothesis Ha={P(ω):ω∈Ωa} can be inferred upon the rejection of null hypothesis Ho={P(ω):ω∈Ω(o)} However, in many cases, tests are carried out and recommendations are made without a precise definition of superiority or a specification of alternative hypothesis. Moreover, in some applications, the union of probability models specified by the chosen null and alternative hypothesis does not constitute a completed model collection P (i.e., H(o)∪H(a) is smaller than P). This not only imposes a strong non-validated assumption of the underlying true models, but also leads to different superiority claims depending on which test is used instead of scientific plausibility. Different ways to partition P fro testing treatment superiority often have different implications on sample size, power, and significance in both efficacy and comparative effectiveness trial design. Such differences are often overlooked. We provide a theoretical framework for evaluating the statistical properties of different specification of superiority in typical hypothesis testing. This can help investigators to select proper hypotheses for treatment comparison inclinical trial design.

A modified Wald interval for the area under the ROC curve (AUC) in diagnostic case-control studies

Background

The area under the receiver operating characteristic (ROC) curve, referred to as the AUC, is an appropriate measure for describing the overall accuracy of a diagnostic test or a biomarker in early phase trials without having to choose a threshold. There are many approaches for estimating the confidence interval for the AUC. However, all are relatively complicated to implement. Furthermore, many approaches perform poorly for large AUC values or small sample sizes.

Methods

The AUC is actually a probability. So we propose a modified Wald interval for a single proportion, which can be calculated on a pocket calculator. We performed a simulation study to compare this modified Wald interval (without and with continuity correction) with other intervals regarding coverage probability and statistical power.

Results

The main result is that the proposed modified Wald intervals maintain and exploit the type I error much better than the intervals of Agresti-Coull, Wilson, and Clopper-Pearson. The interval suggested by Bamber, the Mann-Whitney interval without transformation and also the interval of the binormal AUC are very liberal. For small sample sizes the Wald interval with continuity has a comparable coverage probability as the LT interval and higher power. For large sample sizes the results of the LT interval and of the Wald interval without continuity correction are comparable.

Conclusions

If individual patient data is not available, but only the estimated AUC and the total sample size, the modified Wald intervals can be recommended as confidence intervals for the AUC. For small sample sizes the continuity correction should be used.

Analysis of Predictive Values Based on Individual Risk Factors in Multi-Modality Trials

The accuracy of diagnostic tests with binary end-points is most frequently measured by sensitivity and specificity. However, from the clinical perspective, the main purpose of a diagnostic agent is to assess the probability of a patient actually being diseased and hence predictive values are more suitable here. As predictive values depend on the pre-test probability of disease, we provide a method to take risk factors influencing the patient's prior probability of disease into account, when calculating predictive values. Furthermore, approaches to assess confidence intervals and a methodology to compare predictive values by statistical tests are presented. Hereby the methods can be used to analyze predictive values of factorial diagnostic trials, such as multi-modality, multi-reader-trials. We further performed a simulation study assessing length and coverage probability for different types of confidence intervals, and we present the R-Package facROC that can be used to analyze predictive values in factorial diagnostic trials in particular. The methods are applied to a study evaluating CT-angiography as a noninvasive alternative to coronary angiography for diagnosing coronary artery disease. Hereby the patients' symptoms are considered as risk factors influencing the respective predictive values.

A Rank-Based Test for Comparison of Multidimensional Outcomes

For comparison of multiple outcomes commonly encountered in biomedical research, Huang et al. (2005) improved O'Brien's (1984) rank-sum tests through the replacement of the ad hoc variance by the asymptotic variance of the test statistics. The improved tests control the Type I error rate at the desired level and gain power when the differences between the two comparison groups in each outcome variable fall into the same direction. However, they may lose power when the differences are in different directions (e.g., some are positive and some are negative). These tests and the popular Bonferroni correction failed to show important significant difference when applied to compare heart rates from a clinical trial to evaluate the effect of a procedure to remove the cardioprotective solution HTK. We propose an alternative test statistic, taking the maximum of the individual rank-sum statistics, which controls the type I error and maintains satisfactory power regardless of the directions of the differences. Simulation studies show the proposed test to be of higher power than other tests in certain alternative parameter space of interest. Furthermore, when used to analyze the heart rates data the proposed test yields more satisfactory results.

A rank-based sample size method for multiple outcomes in clinical trials

O'Brien (Biometrics 1984; 40:1079-1087) introduced a rank-sum-type global statistical test to summarize treatment's effect on multiple outcomes and to determine whether a treatment is better than others. This paper presents a sample size computation method for clinical trial design with multiple primary outcomes, and O'Brien's test or its modified test (Biometrics 2005; 61:532-539) is used for the primary analysis. A new measure, the global treatment effect (GTE), is introduced to summarize treatment's efficacy from multiple primary outcomes. Computation of the GTE under various settings is provided. Sample size methods are presented based on prespecified GTE both when pilot data are available and when no pilot data are available. The optimal randomization ratio is given for both cases. We compare our sample size method with the Bonferroni adjustment for multiple tests. Since ranks are used in our derivation, sample size formulas derived here are invariant to any monotone transformation of the data and are robust to outliers and skewed distributions. When all outcomes are binary, we show how sample size is affected by the success probabilities of outcomes. Simulation shows that these sample size formulas provide good control of type I error and statistical power. An application to a Parkinson's disease clinical trial design is demonstrated. Splus codes to compute sample size and the test statistic are provided.

Improving Lesion-Symptom Mapping

Measures of brain activation (e.g., changes in scalp electrical potentials) have become the most popular method for inferring brain function. However, examining brain disruption (e.g., examining behavior after brain injury) can complement activation studies. Activation techniques identify regions involved with a task, whereas disruption techniques are able to discover which regions are crucial for a task. Voxel-based lesion mapping can be used to determine relationships between behavioral measures and the location of brain injury, revealing the function of brain regions. Lesion mapping can also correlate the effectiveness of neurosurgery with the location of brain resection, identifying optimal surgical targets. Traditionally, voxel-based lesion mapping has employed the chi-square test when the clinical measure is binomial and the Student's t test when measures are continuous. Here we suggest that the Liebermeister approach for binomial data is more sensitive than the chi-square test. We also suggest that a test described by Brunner and Munzel is more appropriate than the t test for nonbinomial data because clinical and neuropsychological data often violate the assumptions of the t test. We test our hypotheses comparing statistical tests using both simulated data and data obtained from a sample of stroke patients with disturbed spatial perception. We also developed software to implement these tests (MRIcron), made freely available to the scientific community.

Rank-order versus mean based statistics for neuroimaging

Traditional analysis of neuroimaging data uses parametric statistics, such as the t-test. These tests are designed to detect mean differences. In fact, even nonparametric techniques such as Statistical non-Parametric Mapping (SnPM) use the mean-based t statistic to measure effect size. We note that these measures may not be particularly sensitive for detecting differences when the mean is not an accurate measure of central tendency--for example if one of the groups is experiencing a ceiling or floor effect (causing a skewed data distribution). Here we introduce a nonparametric approach for neuroimaging data analysis that is based on the rank-order of data (and is therefore less influenced by outliers than the t-test). We suggest that this approach may offer a small benefit for datasets where the assumptions of the t-test have been violated, for example datasets where data from one of the groups exhibits a skewed distribution due to floor or ceiling effects.

Adjusting O'Brien's test to control type I error for the generalized nonparametric Behrens-Fisher problem

O'Brien (1984, Biometrics 40, 1079-1087) introduced a simple nonparametric test procedure for testing whether multiple outcomes in one treatment group have consistently larger values than outcomes in the other treatment group. We first explore the theoretical properties of O'Brien's test. We then extend it to the general nonparametric Behrens-Fisher hypothesis problem when no assumption is made regarding the shape of the distributions. We provide conditions when O'Brien's test controls its error probability asymptotically and when it fails. We also provide adjusted tests when the conditions do not hold. Throughout this article, we do not assume that all outcomes are continuous. Simulations are performed to compare the adjusted tests to O'Brien's test. The difference is also illustrated using data from a Parkinson's disease clinical trial.

Nonparametric methods for analysing the accuracy of diagnostic tests with multiple readers

The evaluation of diagnostic agents or imaging procedures is governed by the same scientific and regulatory rules as that of other medical products. Receiver operating characteristic (ROC) curves, and especially the area under these ROC curves, are indices for the accuracy of a diagnostic test for continuous as well as ordinal data. The methodology of multivariate rank statistics for the nonparametric Behrens-Fisher problem is used to evaluate the accuracy of a diagnostic test in a complex factorial design with repeated measurements. Hypotheses are formulated by means of relative treatment effects and are tested by a multivariate extension of the Mann-Whitney statistic in a heteroscedastic model. The application of this method is demonstrated by the analysis of a data set from a diagnostic clinical trial.