The Effect of Two-Stage Sampling on Ordinary Least Squares Methods

Quantitative and Rapid In Vivo Imaging of Human Lenticular Fluorescence

Purpose

To quantitatively investigate the chemical origins of near-UV excited fluorescence in the crystalline lens, and demonstrate the potential usefulness of a rapid and noninvasive diagnostic approach for screening and monitoring of lens damage.

Methods

Anterior segment UV fluorescence imaging was applied to a population of 30 healthy adults, ages 18 to 64 years. Absolute fluorescence intensities and intensity ratios were compared across the population as a function of age. Fluorescence quantum yield (FQY) was calculated from imaging results based on a previous radiometric characterization.

Results

Typical FQYs at 365 nm excitation are approximately 0.2% for healthy adults. Intensity and FQY were observed to increase significantly with age, consistent with ex vivo and confocal microscopy studies. The ratio of blue to green fluorescence is strongly correlated with FQY and age, suggesting that both increases in fluorophore concentration and changes in composition occur with age. Fluorescence data is quantitatively and qualitatively consistent with pyridine nucleotides in young adults, and changes with age are consistent with formation of β-carbolines or advanced glycation end products. Intralens variation is consistent with increased oxidation or glycation in the lens nucleus relative to the cortex.

Conclusions

Lenticular fluorescence can be measured rapidly, accurately, and quantitatively in vivo which provides a spatially resolved, quantitative measure of lens chemistry, including damage from oxidation and glycation. Careful interpretation of fluorescence intensities and intensity ratios can provide chemical insight into lens health. Anterior segment UV fluorescence imaging can thus serve as a useful tool for screening, monitoring, and research of lens damage and cataract formation.

Understanding the Consequences of Collinearity for Multilevel Models: The Importance of Disaggregation Across Levels

In multilevel models, disaggregating predictors into level-specific parts (typically accomplished via centering) benefits parameter estimates and their interpretations. However, the importance of level-specificity has been sparsely addressed in multilevel literature concerning collinearity. In this study, we develop novel insights into the interactivity of centering and collinearity in multilevel models. After integrating the broad literatures on centering and collinearity, we review level-specific and conflated correlations in multilevel data. Next, by deriving formal relationships between predictor collinearity and multilevel model estimates, we demonstrate how the consequences of collinearity change across different centering specifications and identify data characteristics that may exacerbate or mitigate those consequences. We show that when all or some level-1 predictors are uncentered, slope estimates can be greatly biased by collinearity. Disaggregation of all predictors eliminates the possibility that fixed effect estimates will be biased due to collinearity alone; however, under some data conditions, collinearity is associated with biased standard errors and random effect (co)variance estimates. Finally, we illustrate the importance of disaggregation for diagnosing collinearity in multilevel data and provide recommendations for the use of level-specific collinearity diagnostics. Overall, the necessity of disaggregation for identifying and managing collinearity's consequences in multilevel models is clarified in novel ways.

Disaggregating level-specific effects in cross-classified multilevel models

In psychology and other fields, data often have a cross-classified structure, whereby observations are nested within multiple types of non-hierarchical clusters (e.g., repeated measures cross-classified by persons and stimuli). This paper discusses ways that, in cross-classified multilevel models, slopes of lower-level predictors can implicitly reflect an ambiguous blend of multiple effects (for instance, a purely observation-level effect as well as a unique between-cluster effect for each type of cluster). The possibility of conflating multiple effects of lower-level predictors is well recognized for non-cross-classified multilevel models, but has not been fully discussed or clarified for cross-classified contexts. Consequently, in published cross-classified modeling applications, this possibility is almost always ignored, and researchers routinely specify models that conflate multiple effects. In this paper, we show why this common practice can be problematic, and show how to disaggregate level-specific effects in cross-classified models. We provide a novel suite of options that include fully cluster-mean-centered, partially cluster-mean-centered, and contextual effect models, each of which provides a unique interpretation of model parameters. We further clarify how to avoid both fixed and random conflation, the latter of which is widely misunderstood even in non-cross-classified models. We provide simulation results showing the possible deleterious impact of such conflation in cross-classified models, and walk through pedagogical examples to illustrate the disaggregation of level-specific effects. We conclude by considering additional model complexities that can arise with cross-classification, providing guidance for researchers in choosing among model specifications, and describing newly available software to aid researchers who wish to disaggregate effects in practice.

On the Common but Problematic Specification of Conflated Random Slopes in Multilevel Models

For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yields a conflated, uninterpretable fixed effect. For MLMs with random slopes, however, we clarify that two different types of slope conflation can occur: that of the fixed component (termed fixed conflation) and that of the random component (termed random conflation). The latter is rarely recognized and not well understood. Here we explain that a model commonly used to disaggregate the fixed component-the contextual effect model with random slopes-troublingly still yields a conflated random component. Negative consequences of such random conflation have not been demonstrated. Here we show that they include erroneous interpretation and inferences about the substantively important extent of between-cluster differences in slopes, including either underestimating or overestimating such slope heterogeneity. Furthermore, we show that this random conflation can yield inappropriate standard errors for fixed effects. To aid researchers in practice, we delineate which types of random slope specifications yield an unconflated random component. We demonstrate the advantages of these unconflated models in terms of estimating and testing random slope variance (i.e., improved power, Type I error, and bias) and in terms of standard error estimation for fixed effects (i.e., more accurate standard errors), and make recommendations for which specifications to use for particular research purposes.

An overview of optimal designs under a given budget in cluster randomized trials with a binary outcome

Cluster randomized trial design may raise financial concerns because the cost to recruit an additional cluster is much higher than to enroll an additional subject in subject-level randomized trials. Therefore, it is desirable to develop an optimal design. For local optimal designs, optimization means the minimum variance of the estimated treatment effect under the total budget. The local optimal design derived from the variance needs the input of an association parameter ρ in terms of a "working" correlation structure R ( ρ ) in the generalized estimating equation models. When the range of ρ instead of an exact value is available, the parameter space is defined as the range of ρ and the design space is defined as enrollment feasibility, for example, the number of clusters or cluster size. For any value ρ within the range, the optimal design and relative efficiency for each design in the design space is obtained. Then, for each design in the design space, the minimum relative efficiency within the parameter space is calculated. MaxiMin design is the optimal design that maximizes the minimum relative efficiency among all designs in the design space. Our contributions are threefold. First, for three common measures (risk difference, risk ratio, and odds ratio), we summarize all available local optimal designs and MaxiMin designs utilizing generalized estimating equation models when the group allocation proportion is predetermined for two-level and three-level parallel cluster randomized trials. We then propose the local optimal designs and MaxiMin designs using the same models when the group allocation proportion is undecided. Second, for partially nested designs, we develop the optimal designs for three common measures under the setting of equal number of subjects per cluster and exchangeable working correlation structure in the intervention group. Third, we create three new Statistical Analysis System (SAS) macros and update two existing SAS macros for all the optimal designs. We provide two examples to illustrate our methods.

X matters too: How the blended slope problem manifests differently in unilevel vs. multilevel models

Aside from multilevel models (MLMs), several analytic approaches are available for handling cluster-induced dependencies. Nevertheless, the literature on MLM alternatives has called less explicit attention to the potential bias in level-1 (L1) slope coefficients resulting from the “blended slope” problem—a problem that arises when dependencies in predictors (Xs) exist and when L1 predictor-outcome (X-Y) relations differ from those at level-2 (L2). As such, applied researchers may be drawing incorrect inferences about their L1 predictor effects when they specify models without considering clustering in Xs. The present paper reviews this “blended slope” problem and uses Monte Carlo simulation to illustrate how the problem manifests more for unilevel models compared with MLMs. In short, analyses of clustered data should always: 1) report outcome and predictor ICCs, 2) cluster-mean center L1 predictors or incorporate L2 aggregate predictors, and 3) employ a model that takes clustered residuals into account.

Diagnostic accuracy of Spanish and English screeners with Spanish and English criterion measures for bilingual students in Grades 1 and 2

The purpose of this study was to examine the diagnostic accuracy of English and Spanish language screeners when predicting reading comprehension outcomes in both languages at the end of Grade 1 and Grade 2. Participants were 1221 Latino/a bilingual students in Grade 1 and 1004 in Grade 2 who were attending bilingual programs in the Pacific Northwest and in Texas. We used ROC curve analyses to calculate the area under the curve (AUC; A) for each measure. The decision thresholds we selected resulted in 71% of all comparisons having accuracy of at least 0.75. Letter naming, decoding, and oral reading fluency in Spanish were accurate in predicting reading risk on criterion measures in Spanish and in English in Grades 1 and 2 (A value of 0.75 or above). English screeners, however, only predicted reading risk on the English criterion measure, but not on the Spanish criterion measure, with a few exceptions. Implications for practice and future research are discussed.

Consequences of ignoring clustering in linear regression

Background

Clustering of observations is a common phenomenon in epidemiological and clinical research. Previous studies have highlighted the importance of using multilevel analysis to account for such clustering, but in practice, methods ignoring clustering are often employed. We used simulated data to explore the circumstances in which failure to account for clustering in linear regression could lead to importantly erroneous conclusions.

Methods

We simulated data following the random-intercept model specification under different scenarios of clustering of a continuous outcome and a single continuous or binary explanatory variable. We fitted random-intercept (RI) and ordinary least squares (OLS) models and compared effect estimates with the “true” value that had been used in simulation. We also assessed the relative precision of effect estimates, and explored the extent to which coverage by 95% confidence intervals and Type I error rates were appropriate.

Results

We found that effect estimates from both types of regression model were on average unbiased. However, deviations from the “true” value were greater when the outcome variable was more clustered. For a continuous explanatory variable, they tended also to be greater for the OLS than the RI model, and when the explanatory variable was less clustered. The precision of effect estimates from the OLS model was overestimated when the explanatory variable varied more between than within clusters, and was somewhat underestimated when the explanatory variable was less clustered. The cluster-unadjusted model gave poor coverage rates by 95% confidence intervals and high Type I error rates when the explanatory variable was continuous. With a binary explanatory variable, coverage rates by 95% confidence intervals and Type I error rates deviated from nominal values when the outcome variable was more clustered, but the direction of the deviation varied according to the overall prevalence of the explanatory variable, and the extent to which it was clustered.

Conclusions

In this study we identified circumstances in which application of an OLS regression model to clustered data is more likely to mislead statistical inference. The potential for error is greatest when the explanatory variable is continuous, and the outcome variable more clustered (intraclass correlation coefficient is ≥ 0.01).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01333-7.

Change in Circulating Undercarboxylated Osteocalcin (ucOCN) Is Associated With Fat Accumulation in HIV-Seropositive Women

BACKGROUND

Bone mineral density loss and fat accumulation are common in people living with HIV. The bone-derived hormone, undercarboxylated osteocalcin (ucOCN) regulates fat metabolism. We investigated the relationship between ucOCN change and body fat change among perimenopausal/postmenopausal HIV-seronegative and HIV-seropositive women on long-term antiretrovirals.

METHODS

Perimenopausal and postmenopausal women enrolled in the Women's Interagency HIV Study MSK substudy underwent trunk and total fat assessment by dual energy x-ray absorptiometry (DXA) at study enrollment (index visit) and again 2 years later. Circulating ucOCN and cOCN were also measured at the index and 2-year visits. The correlation between the 2-year change in ucOCN and cOCN and change in trunk and total fat was assessed as a function of HIV serostatus using linear regression modeling. Multivariate linear regression assessed the association between ucOCN and cOCN change and total and trunk fat change after adjusting for sociodemographic variables. Linear regression models restricted to HIV-seropositive women were performed to examine the contributions of HIV-specific factors (index CD4 count, viral load, and combined antiretroviral therapy use) on the associations.

RESULTS

Increased ucOCN over the 2-year follow-up was associated with less trunk and total fat accumulation in models adjusting for HIV serostatus and participants sociodemographics, whereas there was no association with cOCN and the fat parameters. None of the HIV-specific factors evaluated influenced the association between ucOCN and fat parameters.

CONCLUSION

The current study suggests that increases in ucOCN are associated with decreased fat accumulation in HIV-seronegative and HIV-seropositive postmenopausal women on long-term antiretroviral therapy.

Machine learning on ligand-residue interaction profiles to significantly improve binding affinity prediction

Structure-based virtual screenings (SBVSs) play an important role in drug discovery projects. However, it is still a challenge to accurately predict the binding affinity of an arbitrary molecule binds to a drug target and prioritize top ligands from an SBVS. In this study, we developed a novel method, using ligand-residue interaction profiles (IPs) to construct machine learning (ML)-based prediction models, to significantly improve the screening performance in SBVSs. Such a kind of the prediction model is called an IP scoring function (IP-SF). We systematically investigated how to improve the performance of IP-SFs from many perspectives, including the sampling methods before interaction energy calculation and different ML algorithms. Using six drug targets with each having hundreds of known ligands, we conducted a critical evaluation on the developed IP-SFs. The IP-SFs employing a gradient boosting decision tree (GBDT) algorithm in conjunction with the MIN + GB simulation protocol achieved the best overall performance. Its scoring power, ranking power and screening power significantly outperformed the Glide SF. First, compared with Glide, the average values of mean absolute error and root mean square error of GBDT/MIN + GB decreased about 38 and 36%, respectively. Second, the mean values of squared correlation coefficient and predictive index increased about 225 and 73%, respectively. Third, more encouragingly, the average value of the areas under the curve of receiver operating characteristic for six targets by GBDT, 0.87, is significantly better than that by Glide, which is only 0.71. Thus, we expected IP-SFs to have broad and promising applications in SBVSs.

The danger of conflating level-specific effects of control variables when primary interest lies in level-2 effects

In the multilevel modelling literature, methodologists widely acknowledge that a level-1 variable can have distinct within-cluster and between-cluster effects, and that failing to disaggregate these can yield a slope estimate that is an uninterpretable, conflated blend of the two. Methodologists have stated, however, that including conflated slopes of level-1 variables in a model is not problematic if substantive interest lies only in effects of level-2 predictors. Researchers commonly follow this advice and use methods that do not disaggregate effects of level-1 control variables (e.g., grand mean centering) when examining effects of level-2 predictors. The primary purpose of this paper is to show that this is a dangerous practice. When level-specific effects of level-1 variables differ, failing to disaggregate them can severely bias estimation of level-2 predictor slopes. We show mathematically why this is the case and highlight factors that can exacerbate such bias. We corroborate these findings with simulations and present an empirical example, showing how such distortions can severely alter substantive conclusions. We ultimately recommend that simply including the cluster mean of the level-1 variable as a control will alleviate the problem.

Using Propensity Score Analysis of Survey Data to Estimate Population Average Treatment Effects: A Case Study Comparing Different Methods

BACKGROUND

Many studies in psychological and educational research aim to estimate population average treatment effects (PATE) using data from large complex survey samples, and many of these studies use propensity score methods. Recent advances have investigated how to incorporate survey weights with propensity score methods. However, to this point, that work had not been well summarized, and it was not clear how much difference the different PATE estimation methods would make empirically.

PURPOSE

The purpose of this study is to systematically summarize the appropriate use of survey weights in propensity score analysis of complex survey data and use a case study to empirically compare the PATE estimates using multiple analysis methods that include ordinary least squares regression, weighted least squares regression, and various propensity score applications.

METHODS

We first summarize various propensity score methods that handle survey weights. We then demonstrate the performance of various analysis methods using a nationally representative data set, the Early Childhood Longitudinal Study-Kindergarten to estimate the effects of preschool on children's academic achievement. The correspondence of the results was evaluated using multiple criteria.

RESULTS AND CONCLUSIONS

It is important for researchers to think carefully about their estimand of interest and use methods appropriate for that estimand. If interest is in drawing inferences to the survey target population, it is important to take the survey weights into account, particularly in the outcome analysis stage for estimating the PATE. The case study shows, however, not much difference among various analysis methods in one applied example.

Repeated Measures Regression in Laboratory, Clinical and Environmental Research: Common Misconceptions in the Matter of Different Within- and between-Subject Slopes

When using repeated measures linear regression models to make causal inference in laboratory, clinical and environmental research, it is typically assumed that the within-subject association of differences (or changes) in predictor variable values across replicates is the same as the between-subject association of differences in those predictor variable values. However, this is often false. For example, with body weight as the predictor variable and blood cholesterol (which increases with higher body fat) as the outcome: (i) a 10-lb weight increase in the same adult affects more greatly an increase in cholesterol in that adult than does (ii) one adult weighing 10 lbs more than a second indicate higher cholesterol in the heavier adult. A 10-lb weight gain in the first adult more likely reflects a build-up of body fat in that person, while a second person being 10 lbs heavier than the first could be influenced by other factors, such as the second person being taller. Hence, to make causal inferences, different within- and between-subject slopes should be separately modeled. A related misconception commonly made using generalized estimation equations (GEE) and mixed models on repeated measures (i.e., for fitting cross-sectional regression) is that the working correlation structure only influences variance of the parameter estimates. However, only independence working correlation guarantees that the modeled parameters have interpretability. We illustrate this with an example where changing working correlation from independence to equicorrelation qualitatively biases parameters of GEE models and show that this happens because within- and between-subject slopes for the outcomes regressed on the predictor variables differ. We then systematically describe several common mechanisms that cause within- and between-subject slopes to differ: change effects, lag/reverse-lag and spillover causality, shared within-subject measurement bias or confounding, and predictor variable measurement error. The misconceptions we describe should be better publicized. Repeated measures analyses should compare within- and between-subject slopes of predictors and when they do differ, investigate the causal reasons for this.

A Bayesian two-stage regression approach of analysing longitudinal outcomes with endogeneity and incompleteness

Abstract:

Two-stage regression methods are typically used for handling endogeneity in the simultaneous equations models in economics and other social sciences. However, the problem is challenging in the presence of incomplete response and/or incomplete endogenous covariate(s). We propose a Bayesian approach for the joint modelling of incomplete longitudinal continuous response and an incomplete count endogenous covariate, where the incompleteness is caused by the censorship through a selection mechanism. We define latent continuous variables which are left-censored at zero and develop a Gibbs sampling algorithm for the simultaneous estimation of the model parameters. We consider partially varying coefficients regression models containing covariates with fixed and time-varying effects on the response. Our work is motivated by a sample dataset from the Health and Retirement Study (HRS) for modelling the out-of-pocket medical cost, where the number of hospital admissions is considered as an endogenous covariate. Our analysis addresses some of the previously unanswered questions on the physical and financial health of the older population based on HRS data. Simulation studies are performed for assessing the usefulness of the proposed method compared to its competitors.

Beta-binomial model for meta-analysis of odds ratios

In meta-analysis of odds ratios (ORs), heterogeneity between the studies is usually modelled via the additive random effects model (REM). An alternative, multiplicative REM for ORs uses overdispersion. The multiplicative factor in this overdispersion model (ODM) can be interpreted as an intra-class correlation (ICC) parameter. This model naturally arises when the probabilities of an event in one or both arms of a comparative study are themselves beta-distributed, resulting in beta-binomial distributions. We propose two new estimators of the ICC for meta-analysis in this setting. One is based on the inverted Breslow-Day test, and the other on the improved gamma approximation by Kulinskaya and Dollinger (2015, p. 26) to the distribution of Cochran's Q. The performance of these and several other estimators of ICC on bias and coverage is studied by simulation. Additionally, the Mantel-Haenszel approach to estimation of ORs is extended to the beta-binomial model, and we study performance of various ICC estimators when used in the Mantel-Haenszel or the inverse-variance method to combine ORs in meta-analysis. The results of the simulations show that the improved gamma-based estimator of ICC is superior for small sample sizes, and the Breslow-Day-based estimator is the best for n⩾100. The Mantel-Haenszel-based estimator of OR is very biased and is not recommended. The inverse-variance approach is also somewhat biased for ORs≠1, but this bias is not very large in practical settings. Developed methods and R programs, provided in the Web Appendix, make the beta-binomial model a feasible alternative to the standard REM for meta-analysis of ORs. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Optimal design of longitudinal data analysis using generalized estimating equation models

Longitudinal studies are often applied in biomedical research and clinical trials to evaluate the treatment effect. The association pattern within the subject must be considered in both sample size calculation and the analysis. One of the most important approaches to analyze such a study is the generalized estimating equation (GEE) proposed by Liang and Zeger, in which "working correlation structure" is introduced and the association pattern within the subject depends on a vector of association parameters denoted by ρ. The explicit sample size formulas for two-group comparison in linear and logistic regression models are obtained based on the GEE method by Liu and Liang. For cluster randomized trials (CRTs), researchers proposed the optimal sample sizes at both the cluster and individual level as a function of sampling costs and the intracluster correlation coefficient (ICC). In these approaches, the optimal sample sizes depend strongly on the ICC. However, the ICC is usually unknown for CRTs and multicenter trials. To overcome this shortcoming, Van Breukelen et al. consider a range of possible ICC values identified from literature reviews and present Maximin designs (MMDs) based on relative efficiency (RE) and efficiency under budget and cost constraints. In this paper, the optimal sample size and number of repeated measurements using GEE models with an exchangeable working correlation matrix is proposed under the considerations of fixed budget, where "optimal" refers to maximum power for a given sampling budget. The equations of sample size and number of repeated measurements for a known parameter value ρ are derived and a straightforward algorithm for unknown ρ is developed. Applications in practice are discussed. We also discuss the existence of the optimal design when an AR(1) working correlation matrix is assumed. Our proposed method can be extended under the scenarios when the true and working correlation matrix are different.

The Australian longitudinal study on male health sampling design and survey weighting: implications for analysis and interpretation of clustered data

BACKGROUND

The Australian Longitudinal Study on Male Health (Ten to Men) used a complex sampling scheme to identify potential participants for the baseline survey. This raises important questions about when and how to adjust for the sampling design when analyzing data from the baseline survey.

METHODS

We describe the sampling scheme used in Ten to Men focusing on four important elements: stratification, multi-stage sampling, clustering and sample weights. We discuss how these elements fit together when using baseline data to estimate a population parameter (e.g., population mean or prevalence) or to estimate the association between an exposure and an outcome (e.g., an odds ratio). We illustrate this with examples using a continuous outcome (weight in kilograms) and a binary outcome (smoking status).

RESULTS

Estimates of a population mean or disease prevalence using Ten to Men baseline data are influenced by the extent to which the sampling design is addressed in an analysis. Estimates of mean weight and smoking prevalence are larger in unweighted analyses than weighted analyses (e.g., mean = 83.9 kg vs. 81.4 kg; prevalence = 18.0 % vs. 16.7 %, for unweighted and weighted analyses respectively) and the standard error of the mean is 1.03 times larger in an analysis that acknowledges the hierarchical (clustered) structure of the data compared with one that does not. For smoking prevalence, the corresponding standard error is 1.07 times larger. Measures of association (mean group differences, odds ratios) are generally similar in unweighted or weighted analyses and whether or not adjustment is made for clustering.

CONCLUSIONS

The extent to which the Ten to Men sampling design is accounted for in any analysis of the baseline data will depend on the research question. When the goals of the analysis are to estimate the prevalence of a disease or risk factor in the population or the magnitude of a population-level exposure-outcome association, our advice is to adopt an analysis that respects the sampling design.

Incremental Exercise, Plasma Concentrations of Catecholamines, Reaction Time, and Motor Time during Performance of a Noncompatible Choice Response Time Task

The primary purpose was to examine the effect of incremental exercise on a noncompatible response time task. Participants ( N = 9) undertook a 4-choice noncompatible response time task under 3 conditions, following rest and during exercise at 70% and 100% of their maximum power output. Reaction and movement times were the dependent variables. Maximum power output had been previously established on an incremental test to exhaustion. A repeated-measures multivariate analysis of variance yielded a significant effect of exercise intensity on the task, but observation of the separate univariate repeated-measures analyses of variance showed that only movement time was significantly affected. Post hoc Tukey tests indicated movement time during maximal intensity exercise was significantly faster than in the other two conditions. The secondary purpose of the study was to assess whether increases in plasma concentrations of adrenaline and nor-adrenaline during exercise and power output would act as predictor variables of reaction and movement times during exercise. Catecholamine concentrations were based on venous blood samples taken during the maximum power output test. None of the variables were significant predictors of reaction time. Only power output was a significant predictor of movement time ( R2 = .24). There was little support for the notion that peripheral concentrations of catecholamines directly induce a central nervous system response.

Meta-analysis of community-based cluster randomization trials with binary outcomes

Background Cluster randomization trials are widely used to test the effect of an intervention when individuals are naturally found in groups such as communities. For several separate studies of a similar intervention, it may be of interest to combine their results using meta-analysis procedures. However, this task requires consideration of both the likely dependencies among cluster members (intracluster correlation) and stratification based on the studies considered.

Purpose In this article, several possible approaches for meta-analysis are considered for cluster randomization trials having a binary outcome.

Methods It is first noted that the standard Mantel—Haenszel test is invalid in this context since it ignores dependencies among cluster members. Two modifications are therefore considered as well as a general inverse variance approach and a procedure based on the Woolf statistic which does not require the availability of trial-specific design effects. Empirical Type I errors and powers for the different procedures considered are evaluated using Monte Carlo simulation. To illustrate the techniques, data are used from trials performed in four countries to compare two antenatal care programs with respect to their effects on the risk of hypertension during pregnancy.

Results For the simulation scenarios considered, an adjusted Mantel—Haenszel procedure provides a valid test with the greatest power slightly outperforming the general inverse variance approach.

Limitations The potential need to adjust for possible confounding was not considered. However, more detailed information on confounders would not likely be available for most meta-analyses.

Conclusion Two procedures performed well. However, the choice of analysis approach also inevitably depends on the nature and extent of the available data. Clinical Trials 2007; 4: 491—498. http://ctj.sagepub.com

Can the buck always be passed to the highest level of clustering?

Background

Clustering commonly affects the uncertainty of parameter estimates in epidemiological studies. Cluster-robust variance estimates (CRVE) are used to construct confidence intervals that account for single-level clustering, and are easily implemented in standard software. When data are clustered at more than one level (e.g. village and household) the level for the CRVE must be chosen. CRVE are consistent when used at the higher level of clustering (village), but since there are fewer clusters at the higher level, and consistency is an asymptotic property, there may be circumstances under which coverage is better from lower- rather than higher-level CRVE. Here we assess the relative importance of adjusting for clustering at the higher and lower level in a logistic regression model.

Methods

We performed a simulation study in which the coverage of 95 % confidence intervals was compared between adjustments at the higher and lower levels.

Results

Confidence intervals adjusted for the higher level of clustering had coverage close to 95 %, even when there were few clusters, provided that the intra-cluster correlation of the predictor was less than 0.5 for models with a single predictor and less than 0.2 for models with multiple predictors.

Conclusions

When there are multiple levels of clustering it is generally preferable to use confidence intervals that account for the highest level of clustering. This only fails if there are few clusters at this level and the intra-cluster correlation of the predictor is high.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0127-1) contains supplementary material, which is available to authorized users.

Internal validation of risk models in clustered data: a comparison of bootstrap schemes

Internal validity of a risk model can be studied efficiently with bootstrapping to assess possible optimism in model performance. Assumptions of the regular bootstrap are violated when the development data are clustered. We compared alternative resampling schemes in clustered data for the estimation of optimism in model performance. A simulation study was conducted to compare regular resampling on only the patient level with resampling on only the cluster level and with resampling sequentially on both the cluster and patient levels (2-step approach). Optimism for the concordance index and calibration slope was estimated. Resampling of only patients or only clusters showed accurate estimates of optimism in model performance. The 2-step approach overestimated the optimism in model performance. If the number of centers or intraclass correlation coefficient was high, resampling of clusters showed more accurate estimates than resampling of patients. The 3 bootstrap schemes also were applied to empirical data that were clustered. The results presented in this paper support the use of resampling on only the clusters for estimation of optimism in model performance when data are clustered.

Estimated long-term fish and shellfish intake--national health and nutrition examination survey

Usual intake estimates describe long-term average intake of food and nutrients and food contaminants. The frequencies of fish and shellfish intake over a 30-day period from National Health and Examination Survey (NHANES 1999-2006) were combined with 24-h dietary recall data from NHANES 2003-2004 using a Monte Carlo procedure to estimate the usual intake of fish and shellfish in this study. Usual intakes were estimated for the US population including children 1 to <11 years, males/females 11 to <16 years, 16 to <21 years, and adults 21+ years. Estimated mean fish intake (consumers only) was highest among children 1 to <2 years and 2 to <3 years, at 0.37 g/kg-day for both age groups, and lowest for females 11 to <16 years, at 0.13 g/kg-day. In all age groups, daily intake estimates were highest for breaded fish, salmon, and mackerel. Among children and teenage consumers, tuna, salmon, and breaded fish were the most frequently consumed fish; shrimp, scallops, and crabs were the most frequently consumed shellfish. The intake estimates from this study better reflect long-term average intake rates and are preferred to assess long-term intake of nutrients and possible exposure to environmental contaminants from fish and shellfish sources than 2-day average estimates.

Prediction models for clustered data: comparison of a random intercept and standard regression model

BACKGROUND

When study data are clustered, standard regression analysis is considered inappropriate and analytical techniques for clustered data need to be used. For prediction research in which the interest of predictor effects is on the patient level, random effect regression models are probably preferred over standard regression analysis. It is well known that the random effect parameter estimates and the standard logistic regression parameter estimates are different. Here, we compared random effect and standard logistic regression models for their ability to provide accurate predictions.

METHODS

Using an empirical study on 1642 surgical patients at risk of postoperative nausea and vomiting, who were treated by one of 19 anesthesiologists (clusters), we developed prognostic models either with standard or random intercept logistic regression. External validity of these models was assessed in new patients from other anesthesiologists. We supported our results with simulation studies using intra-class correlation coefficients (ICC) of 5%, 15%, or 30%. Standard performance measures and measures adapted for the clustered data structure were estimated.

RESULTS

The model developed with random effect analysis showed better discrimination than the standard approach, if the cluster effects were used for risk prediction (standard c-index of 0.69 versus 0.66). In the external validation set, both models showed similar discrimination (standard c-index 0.68 versus 0.67). The simulation study confirmed these results. For datasets with a high ICC (≥15%), model calibration was only adequate in external subjects, if the used performance measure assumed the same data structure as the model development method: standard calibration measures showed good calibration for the standard developed model, calibration measures adapting the clustered data structure showed good calibration for the prediction model with random intercept.

CONCLUSION

The models with random intercept discriminate better than the standard model only if the cluster effect is used for predictions. The prediction model with random intercept had good calibration within clusters.

Outcome-dependent sampling for longitudinal binary response data based on a time-varying auxiliary variable

Outcome-dependent sampling (ODS) study designs are commonly implemented with rare diseases or when prospective studies are infeasible. In longitudinal data settings, when a repeatedly measured binary response is rare, an ODS design can be highly efficient for maximizing statistical information subject to resource limitations that prohibit covariate ascertainment of all observations. This manuscript details an ODS design where individual observations are sampled with probabilities determined by an inexpensive, time-varying auxiliary variable that is related but is not equal to the response. With the goal of validly estimating marginal model parameters based on the resulting biased sample, we propose a semi-parametric, sequential offsetted logistic regressions (SOLR) approach. The SOLR strategy first estimates the relationship between the auxiliary variable and the response and covariate data by using an offsetted logistic regression analysis where the offset is used to adjust for the biased design. Results from the auxiliary variable model are then combined with the known or estimated sampling probabilities to formulate a second offset that is used to correct for the biased design in the ultimate target model relating the longitudinal binary response to covariates. Because the target model offset is estimated with SOLR, we detail asymptotic standard error estimates that account for uncertainty associated with the auxiliary variable model. Motivated by an analysis of the BioCycle Study (Gaskins et al., Effect of daily fiber intake on reproductive function: the BioCycle Study. American Journal of Clinical Nutrition 2009; 90(4): 1061-1069) that aims to describe the relationship between reproductive health (determined by luteinizing hormone levels) and fiber consumption, we examine properties of SOLR estimators and compare them with other common approaches.

Testing logistic regression coefficients with clustered data and few positive outcomes

Applications frequently involve logistic regression analysis with clustered data where there are few positive outcomes in some of the independent variable categories. For example, an application is given here that analyzes the association of asthma with various demographic variables and risk factors using data from the third National Health and Nutrition Examination Survey, a weighted multi stage cluster sample. Although there are 742 asthma cases in all (out of 18,395 individuals), for one of the categories of one of the independent variables there are only 25 asthma cases (out of 695 individuals). Generalized Wald and score hypothesis tests, which use appropriate cluster-level variance estimators, and a bootstrap hypothesis test have been proposed for testing logistic regression coefficients with cluster samples. When there are few positive outcomes, simulations presented in this paper show that these tests can sometimes have either inflated or very conservative levels. A simulation-based method is proposed for testing logistic regression coefficients with cluster samples when there are few positive outcomes. This testing methodology is shown to compare favorably with the generalized Wald and score tests and the bootstrap hypothesis test in terms of maintaining nominal levels. The proposed method is also useful when testing goodness-of-fit of logistic regression models using deciles-of-risk tables.

Short-duration, high-intensity exercise and performance of a sports-specific skill: a preliminary study

The purpose of this study was to examine the effect of one bout and three intermittent bouts of short-duration, high-intensity running on the performance of a sports-specific psychomotor skill. Participants (N=13) were male soccer players (M age 20.5 yr., SD=2.0) who had been playing semi-professionally for M=2.1 years, SD=1.11 and trained twice a week. They undertook a soccer-passing test in three conditions: following rest, following a 100-m sprint and following 3 x 100-m sprints, with 30-sec. rest intervals between sprints. Passing accuracy showed a significant linear deterioration, while number of passes showed a significant quadratic effect. Low to moderate linear regression correlations were found between posttest heart rate and absolute and variable errors on the test. It was concluded that short-duration, high-intensity exercise has a negative effect on accuracy in a sports-specific task that requires both perceptual judgment and motor control.

The merits of breaking the matches: a cautionary tale

Matched-pair cluster randomization trials are frequently adopted as the design of choice for evaluating an intervention offered at the community level. However, previous research has demonstrated that a strategy of breaking the matches and performing an unmatched analysis may be more efficient than performing a matched analysis on the resulting data, particularly when the total number of communities is small and the matching is judged as relatively ineffective. The research concerning this question has naturally focused on testing the effect of intervention. However, a secondary objective of many community intervention trials is to investigate the effect of individual-level risk factors on one or more outcome variables. Focusing on the case of a continuous outcome variable, we show that the practice of performing an unmatched analysis on data arising from a matched-pair design can lead to bias in the estimated regression coefficient, and a corresponding test of significance which is overly liberal. However, for large-scale community intervention trials, which typically recruit a relatively small number of large clusters, such an analysis will generally be both valid and efficient. We also consider other approaches to testing the effect of an individual-level risk factor in a matched-pair cluster randomization design, including a generalized linear model approach that preserves the matching, a two-stage cluster-level analysis, and an approach based on generalized estimating equations.

Environmental correlates of gambling behavior in urban adolescents

The present study considered the relation between adolescent gambling behavior and the perceived environment, the component of Jessor and Jessor's (1977) Problem Behavior Theory that assesses the ways that adolescents perceive the attitudes and behaviors of parents and peers. The predominantly African-American sample included 188 sophomores from two urban public high schools. Using the South Oaks Gambling Screen-Revised for Adolescents to assess gambling risk, rates of both at-risk (20.7%) and problem (12.8%) gambling were found to be high. Boys displayed more gambling problems than did girls. The perceived environment accounted for significant variance in gambling problems and frequency, with proximal components displaying stronger relationships than distal components. Perceiving parent gambling and friend models for problem behavior were positively correlated with gambling problems, and friend models were positively related to gambling frequency. Among girls, family support was positively related to gambling problems. Among boys, this relation was negative.

Symptoms of internalizing and externalizing problems: modeling recovery curves after the death of a parent

BACKGROUND

The death of a parent is a major family disruption that can place children at risk for later depression and other mental health problems.

DESIGN

Theoretically based randomized controlled trial for parentally bereaved children.

SETTING/PARTICIPANTS

Two-hundred and forty-four children and adolescents and their caregivers from 156 families were randomly assigned to the Family Bereavement Program (FBP) intervention condition (90 families; 135 children) or to a control condition (66 families; 109 children). Data collection occurred from 1996 to 1998.

INTERVENTION

Children and caregivers in the intervention condition met separately for 12 two-hour weekly sessions. Skills targeted by the program for children included positive coping, stress appraisals, control beliefs, and self-esteem. The caregiver program targeted caregiver mental health, life stressors, and improved discipline in the home. Both child and caregiver programs focused on improved quality of the caregiver-child relationship.

MAIN OUTCOME MEASURES

Child and caregiver reports of internalizing and externalizing symptoms.

RESULTS

Longitudinal growth curve modeling was performed to model symptoms over time from the point of parental death. The rate of recovery for girls in the program condition was significantly different from that of girls in the control condition across all outcomes. Boys in both conditions showed reduced symptoms over time.

CONCLUSIONS

The methodology offers a conceptually unique way of assessing recovery in terms of reduced mental health problems over time after an event and has contributed to further understanding of FBP intervention effects. The intervention program facilitated recovery among girls, who did not show reduction in behavior problems without the program, while boys demonstrated decreased symptoms even without intervention.

A multilevel approach to the relationship between birth order and intelligence

Many studies show relationships between birth order and intelligence but use cross-sectional designs or manifest other threats to internal validity. Multilevel analyses with a control variable show that when these threats are removed, two major results emerge: (a) birth order has no significant influence on children's intelligence and (b) earlier reported birth order effects on intelligence are attributable to factors that vary between, not within, families. Analyses on 7- to 8 - and 13- to 14-year-old children from the National Longitudinal Survey of Youth support these conclusions. When hierarchical data structures, age variance of children, and within-family versus between-family variance sources are taken into account, previous research is seen in a new light.

Regression models for twin studies: a critical review

Twin studies have long been recognized for their value in learning about the aetiology of disease and specifically for their potential for separating genetic effects from environmental effects. The recent upsurge of interest in life-course epidemiology and the study of developmental influences on later health has provided a new impetus to study twins as a source of unique insights. Twins are of special interest because they provide naturally matched pairs where the confounding effects of a large number of potentially causal factors (such as maternal nutrition or gestation length) may be removed by comparisons between twins who share them. The traditional tool of epidemiological 'risk factor analysis' is the regression model, but it is not straightforward to transfer standard regression methods to twin data, because the analysis needs to reflect the paired structure of the data, which induces correlation between twins. This paper reviews the use of more specialized regression methods for twin data, based on generalized least squares or linear mixed models, and explains the relationship between these methods and the commonly used approach of analysing within-twin-pair difference values. Methods and issues of interpretation are illustrated using an example from a recent study of the association between birth weight and cord blood erythropoietin. We focus on the analysis of continuous outcome measures but review additional complexities that arise with binary outcomes. We recommend the use of a general model that includes separate regression coefficients for within-twin-pair and between-pair effects, and provide guidelines for the interpretation of estimates obtained under this model.

Regression analysis of overdispersed correlated count data with subject specific covariates

A robust likelihood approach for the analysis of overdispersed correlated count data that takes into account cluster varying covariates is proposed. We emphasise two characteristics of the proposed method: That the correlation structure satisfies the constraints on the second moments and that the estimation of the correlation structure guarantees consistent estimates of the regression coefficients. In addition we extend the mean specification to include within- and between-cluster effects. The method is illustrated through the analysis of data from two studies. In the first study, cross-sectional count data from a randomised controlled trial are analysed to evaluate the efficacy of a communication skills training programme. The second study involves longitudinal count data which represent counts of damaged hand joints in patients with psoriatic arthritis. Motivated by this study, we generalize our model to accommodate for a subpopulation of patients who are not susceptible to the development of damaged hand joints.

A discussion of some statistical methods for separating within-pair associations from associations among all twins in research on fetal origins of disease

Twin data can be used to gain insights into the origin of associations between factors arising in fetal life and the risk of later disease. This is because twin data afford an opportunity to conduct paired analyses that take the influence of shared factors into account. When an association that is present in an unpaired analysis is present also in a paired analysis, there is evidence that the causal pathway linking the fetal factor and the disease may have a fetal origin. If the association disappears in the paired analysis, there is evidence that it may have has arisen from a shared source such as the mother. The relevant factors include diet and socio-economic status. There are several statistical approaches to this. The simplest involves comparing, say, a coefficient from a regression of an outcome on a fetal factor for all subjects in a twin sample, with the coefficient obtained from regressing the within-pair difference in the outcome on the within-pair difference in the fetal factor. Alternative approaches involve simultaneously estimating regression parameters for between- and within-pair components. These approaches permit similar inferences about whether the association is due to individual (fetal) or shared (maternal) factors, and are valid in the circumstances that non-shared factors missing from the regression model do not influence the regression estimates.

Alternative methods for handling attrition: an illustration using data from the Fast Track evaluation

Using data from the evaluation of the Fast Track intervention, this article illustrates three methods for handling attrition. Multiple imputation and ignorable maximum likelihood estimation produce estimates that are similar to those based on listwise-deleted data. A panel selection model that allows for selective dropout reveals that highly aggressive boys accumulate in the treatment group over time and produces a larger estimate of treatment effect. In contrast, this model produces a smaller treatment effect for girls. The article's conclusion discusses the strengths and weaknesses of the alternative approaches and outlines ways in which researchers might improve their handling of attrition.

Sample size considerations in observational health care quality studies

A common objective in health care quality studies involves measuring and comparing the quality of care delivered to cohorts of patients by different health care providers. The data used for inference involve observations on units grouped within clusters, such as patients treated within hospitals. Unlike cluster randomization trials where often clusters are randomized to interventions to learn about individuals, the target of inference in health quality studies is the cluster. Furthermore, randomization is often not performed and the resulting biases may invalidate standard tests. In this paper, we discuss approaches to sample size determination in the design of observational health quality studies when the outcome is binary. Methods for calculating sample size using marginal models are briefly reviewed, but the focus is on hierarchical binomial models. Sample size in unbalanced clusters and stratified designs are characterized. We draw upon the experiences that have arisen from a study funded by the Agency for Healthcare Research and Quality involving assessment of quality of care for patients with cardiovascular disease. If researchers are interested in comparing clusters, hierarchical models are preferred.

Multilevel Models and Unbiased Tests for Group Based Interventions: Examples from the Safer Choices Study

For many large-scale behavioral interventions, random assignment to intervention condition occurs at the group level. Data analytic models that ignore potential non-independence of observations provide inefficient parameter estimates and often produce biased test statistics. For studies in which individuals are randomized by groups to treatment condition, multilevel models (MLMs) provide a flexible approach to statistically evaluating program effects. This article presents an explanation of the need for MLM's for such nested designs and uses data from the Safer Choices study to illustrate the application of MLMs for both continuous and dichotomous outcomes. When designing studies, researchers who are considering group-randomized interventions should also consider the features of the multilevel analytic models they might employ.

Multilevel Modeling of Individual and Group Level Mediated Effects

This article combines procedures for single-level mediational analysis with multilevel modeling techniques in order to appropriately test mediational effects in clustered data. A simulation study compared the performance of these multilevel mediational models with that of single-level mediational models in clustered data with individual- or group-level initial independent variables, individual- or group-level mediators, and individual level outcomes. The standard errors of mediated effects from the multilevel solution were generally accurate, while those from the single-level procedure were downwardly biased, often by 20% or more. The multilevel advantage was greatest in those situations involving group-level variables, larger group sizes, and higher intraclass correlations in mediator and outcome variables. Multilevel mediational modeling methods were also applied to data from a preventive intervention designed to reduce intentions to use steroids among players on high school football teams. This example illustrates differences between single-level and multilevel mediational modeling in real-world clustered data and shows how the multilevel technique may lead to more accurate results.