Inferences on the common mean of several normal populations based on the generalized variable method

Optimal Cut-Point Selection Methods Under Binary Classification When Subclasses Are Involved

In practice, we often encounter binary classification problems where both main classes consist of multiple subclasses. For example, in an ovarian cancer study where biomarkers were evaluated for their accuracy of distinguishing noncancer cases from cancer cases, the noncancer class consists of healthy subjects and benign cases, while the cancer class consists of subjects at both early and late stages. This article aims to provide a large number of optimal cut-point selection methods for such setting. Furthermore, we also study confidence interval estimation of the optimal cut-points. Simulation studies are carried out to explore the performance of the proposed cut-point selection methods as well as confidence interval estimation methods. A real ovarian cancer data set is analyzed using the proposed methods.

Evaluating joint confidence region of hypervolume under ROC manifold and generalized Youden index

In biomarker evaluation/diagnostic studies, the hypervolume under the receiver operating characteristic manifold (HUM K $$ {\mathrm{HUM}}_K $$ ) and the generalized Youden index (J K $$ {J}_K $$ ) are the most popular measures for assessing classification accuracy under multiple classes. WhileHUM K $$ {\mathrm{HUM}}_K $$ is frequently used to evaluate the overall accuracy,J K $$ {J}_K $$ provides direct measure of accuracy at the optimal cut-points. Simultaneous evaluation ofHUM K $$ {\mathrm{HUM}}_K $$ andJ K $$ {J}_K $$ provides a comprehensive picture about the classification accuracy of the biomarker/diagnostic test under consideration. This article studies both parametric and non-parametric approaches for estimating the confidence region ofHUM K $$ {\mathrm{HUM}}_K $$ andJ K $$ {J}_K $$ for a single biomarker. The performances of the proposed methods are investigated by an extensive simulation study and are applied to a real data set from the Alzheimer's Disease Neuroimaging Initiative.

Confidence estimation based on data from independent studies

The problem of finding confidence intervals based on data from several independent studies or experiments is considered. A general method of finding confidence intervals by inverting a combined test is proposed. The combined tests considered are the Fisher test, the weighted inverse normal test, the inverse chi-square test and the inverse Cauchy test. The method is illustrated for finding confidence intervals for a common mean of several normal populations, common correlation coefficient of several bivariate normal populations, common coefficient of variation, common mean of several lognormal populations, and for a common mean of several gamma populations. For each case, the confidence intervals based on the combined tests are compared with the other available approximate confidence intervals with respect to coverage probability and precision. R functions to compute all confidence intervals are provided in a supplementary file. The methods are illustrated using several practical examples.

A new accuracy metric under three classes when subclasses are involved and its confidence interval estimation

"Compound multi-class classification" refers to the setting where three or more main classes are involved and at least one of the main classes have multiple subclasses. A common practice in evaluating biomarker performance under "compound multi-class classification" is "subclasses pooling." In this article, we first explore the downsides of accuracy metrics based on pooled data. Then we propose a new accuracy measure proper for "compound multi-class classification" with three ordinal main classes, namely "volume under compoundR O C $$ ROC $$ surface (V U S C $$ VU{S}_C $$ )." The proposedV U S C $$ VU{S}_C $$ evaluates the accuracy of a biomarker appropriately by identifying main classes without requiring specification of an ordering for marker values of subclasses within each main class. For confidence interval estimation ofV U S C $$ VU{S}_C $$ , both parametric and nonparametric methods are studied, and simulation studies are carried out to assess coverage probabilities. A subset of Alzheimer's Disease Neuroimaging Initiative study dataset is analyzed.

Estimating average wind speed in Thailand using confidence intervals for common mean of several Weibull distributions

The Weibull distribution has been used to analyze data from many fields, including engineering, survival and lifetime analysis, and weather forecasting, particularly wind speed data. It is useful to measure the central tendency of wind speed data in specific locations using statistical parameters for instance the mean to accurately forecast the severity of future catastrophic events. In particular, the common mean of several independent wind speed samples collected from different locations is a useful statistic. To explore wind speed data from several areas in Surat Thani province, a large province in southern Thailand, we constructed estimates of the confidence interval for the common mean of several Weibull distributions using the Bayesian equitailed confidence interval and the highest posterior density interval using the gamma prior. Their performances are compared with those of the generalized confidence interval and the adjusted method of variance estimates recovery based on their coverage probabilities and expected lengths. The results demonstrate that when the common mean is small and the sample size is large, the Bayesian highest posterior density interval performed the best since its coverage probabilities were higher than the nominal confidence level and it provided the shortest expected lengths. Moreover, the generalized confidence interval performed well in some scenarios whereas adjusted method of variance estimates recovery did not. The approaches were used to estimate the common mean of real wind speed datasets from several areas in Surat Thani province, Thailand, fitted to Weibull distributions. These results support the simulation results in that the Bayesian methods performed the best. Hence, the Bayesian highest posterior density interval is the most appropriate method for establishing the confidence interval for the common mean of several Weibull distributions.

Estimation of common percentile of rainfall datasets in Thailand using delta-lognormal distributions

Weighted percentiles in many areas can be used to investigate the overall trend in a particular context. In this article, the confidence intervals for the common percentile are constructed to estimate rainfall in Thailand. The confidence interval for the common percentile help to indicate intensity of rainfall. Herein, four new approaches for estimating confidence intervals for the common percentile of several delta-lognormal distributions are presented: the fiducial generalized confidence interval, the adjusted method of variance estimates recovery, and two Bayesian approaches using fiducial quantity and approximate fiducial distribution. The Monte Carlo simulation was used to evaluate the coverage probabilities and average lengths via the R statistical program. The proposed confidence intervals are compared in terms of their coverage probabilities and average lengths, and the results of a comparative study based on these metrics indicate that one of the Bayesian confidence intervals is better than the others. The efficacies of the approaches are also illustrated by applying them to daily rainfall datasets from various regions in Thailand.

Confidence interval estimation of the common mean of several gamma populations

Gamma distributions are widely used in applied fields due to its flexibility of accommodating right-skewed data. Although inference methods for a single gamma mean have been well studied, research on the common mean of several gamma populations are sparse. This paper addresses the problem of confidence interval estimation of the common mean of several gamma populations using the concept of generalized inference and the method of variance estimates recovery (MOVER). Simulation studies demonstrate that several proposed approaches can provide confidence intervals with satisfying coverage probabilities even at small sample sizes. The proposed methods are illustrated using two examples.

Joint inference about the AUC and Youden index for paired biomarkers

It is common to compare biomarkers' diagnostic or prognostic performance using some summary ROC measures such as the area under the ROC curve (AUC) or the Youden index. We propose to compare two paired biomarkers using both the AUC and the Youden index since the two indices describe different aspects of the ROC curve. This comparison can be made by estimating the joint confidence region (an elliptical area) of the differences of the paired AUCs and the Youden indices. Furthermore, for deciding if one marker is better than the other in terms of both the A U C and the Youden index (J), we can test H 0 : A U C a ≤ A U C b or J a ≤ J b against H a : A U C a > A U C b and J a > J b using the paired differences. The construction of such a joint hypothesis is an example of the multivariate order-restricted hypotheses. For such a hypothesis, we propose and compare three testing procedures: (1) the intersection-union test ( I U T ); (2) the conditional test; and (3) the joint test. The performance of the proposed inference methods was evaluated and compared through simulations. The simulation results demonstrate that the proposed joint confidence region maintains the desired confidence level, and all three tests maintain the type I error under the null. Furthermore, among the three proposed testing methods, the conditional test is the preferred approach with markedly larger power consistently than the other two competing methods.

Joint confidence region estimation on predictive values

For evaluating diagnostic accuracy of inherently continuous diagnostic tests/biomarkers, sensitivity and specificity are well-known measures both of which depend on a diagnostic cut-off, which is usually estimated. Sensitivity (specificity) is the conditional probability of testing positive (negative) given the true disease status. However, a more relevant question is "what is the probability of having (not having) a disease if a test is positive (negative)?". Such post-test probabilities are denoted as positive predictive value (PPV) and negative predictive value (NPV). The PPV and NPV at the same estimated cut-off are correlated, hence it is desirable to make the joint inference on PPV and NPV to account for such correlation. Existing inference methods for PPV and NPV focus on the individual confidence intervals and they were developed under binomial distribution assuming binary instead of continuous test results. Several approaches are proposed to estimate the joint confidence region as well as the individual confidence intervals of PPV and NPV. Simulation results indicate the proposed approaches perform well with satisfactory coverage probabilities for normal and non-normal data and, additionally, outperform existing methods with improved coverage as well as narrower confidence intervals for PPV and NPV. The Alzheimer's Disease Neuroimaging Initiative (ADNI) data set is used to illustrate the proposed approaches and compare them with the existing methods.

Confidence interval estimation for sensitivity and difference between two sensitivities at a given specificity under tree ordering

This article considers a setting in diagnostic studies (or biomarker study) which involves a healthy class and a diseased class and the latter consists of several subclasses. The problem of interest is to evaluate the accuracy of a biomarker (or a diagnostic test) measured on a continuous scale correctly identifying healthy subjects from diseased subjects without requiring specification of an ordering in terms of marker values for subclasses relative to each other within the diseased class. Such setting is quite common in practice and it falls in the framework of tree ordering or umbrella ordering. This article explores several parametric and nonparametric approaches for estimating confidence intervals of sensitivity of single biomarker and difference between sensitivities of two correlated biomarkers under tree ordering at a given specificity. The performances of all the methods are evaluated and compared by a comprehensive simulation study. A published microarray data set is analyzed using the proposed methods.

Confidence intervals for the common coefficient of variation of rainfall in Thailand

The log-normal distribution is often used to analyze environmental data like daily rainfall amounts. The rainfall is of interest in Thailand because high variable climates can lead to periodic water stress and scarcity. The mean, standard deviation or coefficient of variation of the rainfall in the area is usually estimated. The climate moisture index is the ratio of plant water demand to precipitation. The climate moisture index should use the coefficient of variation instead of the standard deviation for comparison between areas with widely different means. The larger coefficient of variation indicates greater dispersion, whereas the lower coefficient of variation indicates the lower risk. The common coefficient of variation, is the weighted coefficients of variation based on k areas, presents the average daily rainfall. Therefore, the common coefficient of variation is used to describe overall water problems of k areas. In this paper, we propose four novel approaches for the confidence interval estimation of the common coefficient of variation of log-normal distributions based on the fiducial generalized confidence interval (FGCI), method of variance estimates recovery (MOVER), computational, and Bayesian approaches. A Monte Carlo simulation was used to evaluate the coverage probabilities and average lengths of the confidence intervals. In terms of coverage probability, the results show that the FGCI approach provided the best confidence interval estimates for most cases except for when the sample case was equal to six populations (k = 6) and the sample sizes were small (n _I  < 50), for which the MOVER confidence interval estimates were the best. The efficacies of the proposed approaches are illustrated with example using real-life daily rainfall datasets from regions of Thailand.

Measuring diagnostic accuracy for biomarkers under tree-ordering

In the field of diagnostic studies for tree or umbrella ordering, under which the marker measurement for one class is lower or higher than those for the rest unordered classes, there exist a few diagnostic measures such as the naive AUC ( NAUC), the umbrella volume ( UV), and the recently proposed TAUC, i.e. area under a ROC curve for tree or umbrella ordering (TROC). However, an important characteristic about tree or umbrella ordering has been neglected. This paper mainly focuses on promoting the use of the integrated false negative rate under tree ordering ( ITFNR) as an additional diagnostic measure besides TAUC, and proposing the idea of using ( TAUC, ITFNR) instead of TAUC to evaluate the diagnostic accuracy of a biomarker under tree or umbrella ordering. Parametric and non-parametric approaches for constructing joint confidence region of ( TAUC, ITFNR) are proposed. Simulation studies under a variety of settings are carried out to assess and compare the performance of these methods. In the end, a published microarray data set is analyzed.

Confidence intervals for differences between volumes under receiver operating characteristic surfaces (VUS) and generalized Youden indices (GYIs)

This article explores both existing and new methods for the construction of confidence intervals for differences of indices of diagnostic accuracy of competing pairs of biomarkers in three-class classification problems and fills the methodological gaps for both parametric and non-parametric approaches in the receiver operating characteristic surface framework. The most widely used such indices are the volume under the receiver operating characteristic surface and the generalized Youden index. We describe implementation of all methods and offer insight regarding the appropriateness of their use through a large simulation study with different distributional and sample size scenarios. Methods are illustrated using data from the Alzheimer's Disease Neuroimaging Initiative study, where assessment of cognitive function naturally results in a three-class classification setting.

Confidence interval estimation of the difference between two sensitivities to the early disease stage

Although most of the statistical methods for diagnostic studies focus on disease processes with binary disease status, many diseases can be naturally classified into three ordinal diagnostic categories, that is normal, early stage, and fully diseased. For such diseases, the volume under the ROC surface (VUS) is the most commonly used index of diagnostic accuracy. Because the early disease stage is most likely the optimal time window for therapeutic intervention, the sensitivity to the early diseased stage has been suggested as another diagnostic measure. For the purpose of comparing the diagnostic abilities on early disease detection between two markers, it is of interest to estimate the confidence interval of the difference between sensitivities to the early diseased stage. In this paper, we present both parametric and non-parametric methods for this purpose. An extensive simulation study is carried out for a variety of settings for the purpose of evaluating and comparing the performance of the proposed methods. A real example of Alzheimer's disease (AD) is analyzed using the proposed approaches.

Joint confidence region estimation for area under ROC curve and Youden index

In the field of diagnostic studies, the area under the ROC curve (AUC) serves as an overall measure of a biomarker/diagnostic test's accuracy. Youden index, defined as the overall correct classification rate minus one at the optimal cut-off point, is another popular index. For continuous biomarkers of binary disease status, although researchers mainly evaluate the diagnostic accuracy using AUC, for the purpose of making diagnosis, Youden index provides an important and direct measure of the diagnostic accuracy at the optimal threshold and hence should be taken into consideration in addition to AUC. Furthermore, AUC and Youden index are generally correlated. In this paper, we initiate the idea of evaluating diagnostic accuracy based on AUC and Youden index simultaneously. As the first step toward this direction, this paper only focuses on the confidence region estimation of AUC and Youden index for a single marker. We present both parametric and non-parametric approaches for estimating joint confidence region of AUC and Youden index. We carry out extensive simulation study to evaluate the performance of the proposed methods. In the end, we apply the proposed methods to a real data set.

Testing equality of generalized treatment effects

Recently, the concept of generalized treatment effect, defined as P(X > Y) where X and Y denote continuous outcome variables for treatment arm and control arm, respectively, has been proposed as an appropriate measure of treatment effect in clinical trials with parallel design. Compared to the mean difference, the generalized treatment effect has many advantages; for example, it is a scaleless measure and it does not change under monotonic transformations. This article investigates the problem of testing equality of generalized treatment effects among several clinical trials. The proposed approach follows the same vein as the generalized variable method for testing equality of several log-normal means proposed by Li (2009). Numerical study demonstrates that the proposed test has excellent type I error control for clinical trials with small to medium sample sizes. Robustness study shows that the proposed method performs reasonably for categorical data.

Parametric and non-parametric confidence intervals of the probability of identifying early disease stage given sensitivity to full disease and specificity with three ordinal diagnostic groups

In practice, there exist many disease processes with three ordinal disease classes, that is, the non-diseased stage, the early disease stage, and the fully diseased stage. Because early disease stage is likely the best time window for treatment interventions, it is important to have diagnostic tests that have good diagnostic ability to discriminate the early disease stage from the other two stages. In this paper, we present both parametric and non-parametric approaches for confidence interval estimation of probability of detecting early disease stage given the true classification rates for non-diseased group and diseased group, namely, the specificity and the sensitivity to full disease. We analyze a data set on the clinical diagnosis of early-stage Alzheimer's disease from the neuropsychological database at the Washington University Alzheimer's Disease Research Center using the proposed approaches.

Exact confidence interval estimation for the Youden index and its corresponding optimal cut-point

In diagnostic studies, the receiver operating characteristic (ROC) curve and the area under the ROC curve are important tools in assessing the utility of biomarkers in discriminating between non-diseased and diseased populations. For classifying a patient into the non-diseased or diseased group, an optimal cut-point of a continuous biomarker is desirable. Youden's index (J), defined as the maximum vertical distance between the ROC curve and the diagonal line, serves as another global measure of overall diagnostic accuracy and can be used in choosing an optimal cut-point. The proposed approach is to make use of a generalized approach to estimate the confidence intervals of the Youden index and its corresponding optimal cut-point. Simulation results are provided for comparing the coverage probabilities of the confidence intervals based on the proposed method with those based on the large sample method and the parametric bootstrap method. Finally, the proposed method is illustrated via an application to a data set from a study on Duchenne muscular dystrophy (DMD).

Exact confidence interval estimation for the difference in diagnostic accuracy with three ordinal diagnostic groups

In the cases with three ordinal diagnostic groups, the important measures of diagnostic accuracy are the volume under surface (VUS) and the partial volume under surface (PVUS) which are the extended forms of the area under curve (AUC) and the partial area under curve (PAUC). This article addresses confidence interval estimation of the difference in paired VUS s and the difference in paired PVUS s. To focus especially on studies with small to moderate sample sizes, we propose an approach based on the concepts of generalized inference. A Monte Carlo study demonstrates that the proposed approach generally can provide confidence intervals with reasonable coverage probabilities even at small sample sizes. The proposed approach is compared to a parametric bootstrap approach and a large sample approach through simulation. Finally, the proposed approach is illustrated via an application to a data set of blood test results of anemia patients.

Confidence interval estimation of the difference between paired AUCs based on combined biomarkers

In many diagnostic studies, multiple diagnostic tests are performed on each subject or multiple disease markers are available. Commonly, the information should be combined to improve the diagnostic accuracy. We consider the problem of comparing the discriminatory abilities between two groups of biomarkers. Specifically, this article focuses on confidence interval estimation of the difference between paired AUCs based on optimally combined markers under the assumption of multivariate normality. Simulation studies demonstrate that the proposed generalized variable approach provides confidence intervals with satisfying coverage probabilities at finite sample sizes. The proposed method can also easily provide P-values for hypothesis testing. Application to analysis of a subset of data from a study on coronary heart disease illustrates the utility of the method in practice.

Inferences about the between-study variance in meta-analysis with normally distributed outcomes

This paper presents a new approach for confidence interval estimation of the between-study variance in meta-analysis with normally distributed responses based on the concepts of generalized variables. Simulation study shows that the coverage probabilities of the proposed confidence intervals are generally satisfactory. Moreover, the proposed approach can easily provide P -values for hypothesis testing. For meta-analysis of controlled clinical trials or epidemiological studies, within which the responses are normally distributed, the proposed approach is an ideal candidate for making inference about the between-study variance.