Preliminary Baseline Vestibular Ocular Motor Screening Scores in Pediatric Soccer Athletes

CONTEXT

The utility of baseline vestibular and ocular motor screening (VOMS) in high school and collegiate athletes is demonstrated throughout the literature; however, baseline VOMS data at the youth level are limited. In addition, with the recent adoption of the change scoring method, there is a need to document baseline VOMS total and change scores in a pediatric population.

OBJECTIVE

To document baseline VOMS total and change scores and to document the internal consistency of the VOMS in pediatric soccer athletes. We hypothesized that the VOMS would demonstrate strong internal consistency in pediatric soccer athletes.

DESIGN

Cross-sectional study.

METHODS

Pediatric soccer athletes (N = 110; range = 5-12 y) completed the VOMS at baseline. Descriptive statistics summarized demographic information, VOMS total scores, and VOMS change scores. Cronbach α assessed internal consistency for VOMS total scores and change scores.

RESULTS

Twenty-one (19.1%) participants had at least one total score above clinical cutoffs (≥2 on any VOMS component and ≥5 cm on average near point convergence). Forty (36.4%) participants had at least one change score above clinical cutoffs (≥1 on any VOMS component and ≥3 cm on average near point convergence). The internal consistency was strong for total scores with all VOMS components included (Cronbach α = .80) and change scores (Cronbach α = .89).

CONCLUSIONS

Although results suggest VOMS items measure distinct components of the vestibular and ocular motor systems, caution should be taken when interpreting VOMS total and change scores in pediatric athletes, as overreporting symptoms is common, thereby impacting the false-positive rate.

Change Score and Subscore Precision and Reliability of the Children's Depression Inventory

The Child Depression Inventory (CDI) is often used to assess change in depression over time, but no studies estimate the reliability of CDI change scores nor its five subscores. Our study investigated the reliability of change scores for both the total score on the CDI as well as its five subscores. We examined CDI responses from 186 maltreated children and estimated change score reliability for relative (e.g., comparison) and absolute (e.g., diagnosis) purposes. We also conducted subscore utility analysis, which determines if subscores have adequate reliability and provide information beyond the total score. We found that the total change score had acceptable reliability of .70 for our sample for both relative and absolute interpretations. In addition, the total score was a better predictor of true subscore values than the observed subscores-suggesting subscores did not add value over the total score, and that the reliability of changes in subscores was too low to be useful for any purpose. In summary, we found that the total CDI change scores were useful for assessing change in studies that examine relative or absolute change, and we advise caution when interpreting CDI subscores based on our analysis.

Special Issue PRO-Analysis of Clinically Meaningful Change on Patient-Reported Outcomes: Renewed Insights About Covariate Adjustment

Determining clinically meaningful change (CMC) in a patient-reported (PRO) measure is central to its existence in gauging how patients feel and function, especially for evaluating a treatment effect. Anchor-based approaches are recommended to estimate a CMC threshold on a PRO measure. Determination of CMC involves linking changes or differences in the target PRO measure to that in an external (anchor) measure that is easier to interpret than and appreciably associated with the PRO measure. One type of anchor-based approach for CMC is the "mean change method" where the mean change in score of the target PRO measure within a particular anchor transition level (e.g. one-category improvement) is subtracted from the mean change in score of within an adjacent anchor category (e.g. no change category). In the literature, the mean change method has been applied with and without an adjustment for the baseline scores for the PRO of interest. This article provides the analytic rationale and conceptual justification for keeping the analysis unadjusted and not controlling for baseline PRO scores. Two illustrative examples are highlighted. The current research is essentially a variation of Lord's paradox (where whether to adjust for a baseline variable depends on the research question) placed in a new context. Once the adjustment is made, the resulting CMC estimate reflects an artificial case where the anchor transition levels are forced to have the same average baseline PRO score. The unadjusted estimate acknowledges that the anchor transition levels are naturally occurring (not randomized) groups and thus maintains external validity.

Emotion dysregulation and eating disorder outcome: Prediction, change and contribution of self-image

OBJECTIVES

Eating disorders (EDs) are severe disorders with unsatisfactory outcome. Emotion dysregulation and self-image are suggested maintenance factors; this study examined emotion dysregulation as potential predictor and/or mechanism of change in relation to ED outcome, and associations between change in emotion dysregulation and self-image in relation to outcome.

DESIGN

Registry data from initial and 1-year follow-up assessments for 307 patients with a wide range of EDs in specialized ED treatment were used.

METHODS

Initial and change (∆) in emotion dysregulation were examined as predictors of 1-year outcome. Direct and indirect associations between ∆emotion dysregulation and ∆self-image as either independent variable or mediator in relation to ∆ED psychopathology as dependent were also examined.

RESULTS

Higher initial emotion dysregulation was weakly associated with higher follow-up ED psychopathology, but not remission, while relative increase in emotion dysregulation was associated with both higher follow-up psychopathology and increased risk of still having a diagnosis. Change in emotion dysregulation primarily had an indirect effect (through change in self-image), while change in self-image had a direct effect, on change in ED psychopathology improvement (such that improvement in one was associated with improvement in the other).

CONCLUSIONS

Results identify emotion dysregulation as a potential mechanism of change in relation to ED outcome. However, this association was mainly mediated by change in self-image. Results indicate that, in order to improve emotion regulation as a means to reduce ED psychopathology, improving self-image is essential.

Vestibular/Ocular Motor Screening Assessment Outcomes After Sport-Related Concussion in High School and Collegiate Athletes

CONTEXT

Vestibular and ocular motor assessment is an emerging clinical assessment for patients with sport-related concussion (SRC). The increased use of these assessments by clinicians calls for the examination of outcomes that may affect clinical practice.

OBJECTIVE

To compare vestibular and ocular motor impairments in high school and collegiate athletes within 72 hours of SRC and examine the distribution of impairments in these populations based on pre-established clinical cutoff scores.

DESIGN

Cross-sectional study.

SETTING

High school and collegiate athletics.

PATIENTS OR OTHER PARTICIPANTS

Data were collected from 110 athletes (high school: n = 47, age = 15.40 ± 1.35 years; college: n = 63, age = 19.46 ± 1.28 years) within 72 hours of sustaining an SRC.

MAIN OUTCOME MEASURE(S)

Total and change scores were calculated for the Vestibular/Ocular Motor Screening (VOMS) tool, along with average near point of convergence (NPC) distance. Separate Mann-Whitney U tests were used to compare group differences, and χ2 analyses were used to examine the proportion of athletes with scores greater than clinical cutoff scores for all VOMS outputs. The α level was set a priori at .05.

RESULTS

No differences were found between high school and collegiate athletes for VOMS total and change scores and NPC distance. A larger proportion of the sample had scores greater than the cutoff for all total scores (P < .001) and change scores in horizontal vestibulo-ocular reflex (59.01%; P < .001), vertical vestibulo-ocular reflex (60.91%; P < .001), and visual motion sensitivity (60.91%; P < .001). However, a larger proportion demonstrated smooth pursuit change scores (85.45%; P < .001) and NPC distances (73.64%; P = .01) that were less than the cutoff scores.

CONCLUSIONS

During the acute phase of SRC, high school and collegiate athletes presented with similar vestibular and ocular motor impairments as measured using the VOMS, but vestibular tasks appeared to cause greater symptom provocation. Lastly, VOMS change scores may offer more clinical utility compared with total scores in assessing specific impairments after SRC.

Analyses of 'change scores' do not estimate causal effects in observational data

Background

In longitudinal data, it is common to create ‘change scores’ by subtracting measurements taken at baseline from those taken at follow-up, and then to analyse the resulting ‘change’ as the outcome variable. In observational data, this approach can produce misleading causal-effect estimates. The present article uses directed acyclic graphs (DAGs) and simple simulations to provide an accessible explanation for why change scores do not estimate causal effects in observational data.

Methods

Data were simulated to match three general scenarios in which the outcome variable at baseline was a (i) ‘competing exposure’ (i.e. a cause of the outcome that is neither caused by nor causes the exposure), (ii) confounder or (iii) mediator for the total causal effect of the exposure variable at baseline on the outcome variable at follow-up. Regression coefficients were compared between change-score analyses and the appropriate estimator(s) for the total and/or direct causal effect(s).

Results

Change-score analyses do not provide meaningful causal-effect estimates unless the baseline outcome variable is a ‘competing exposure’ for the effect of the exposure on the outcome at follow-up. Where the baseline outcome is a confounder or mediator, change-score analyses evaluate obscure estimands, which may diverge substantially in magnitude and direction from the total and direct causal effects.

Conclusion

Future observational studies that seek causal-effect estimates should avoid analysing change scores and adopt alternative analytical strategies.

Difference-in-Difference in the Time of Cholera: a Gentle Introduction for Epidemiologists

Purpose of review

The goal of this article is to provide an introduction to the intuition behind the difference-in-difference method for epidemiologists. We focus on the theoretical aspects of this tool, including the types of questions for which difference-in-difference is appropriate, and what assumptions must hold for the results to be causally interpretable.

Recent findings

While currently under-utilized in epidemiologic research, the difference-in-difference method is a useful tool to examine effects of population-level exposures, but relies on strong assumptions.

Summary

We use the famous example of John Snow's investigation of the cause of cholera mortality in London to illustrate the difference-in-difference approach and corresponding assumptions. We conclude by arguing that this method deserves a second-look from epidemiologists interested in asking causal questions about the impact of a population-level exposure change on a population-level outcome for the group that experienced the change.

Analysis of change in patient-reported outcome measures with floor and ceiling effects using the multilevel Tobit model: a simulation study and an example from a National Joint Register using body mass index and the Oxford Hip Score

OBJECTIVES

This study has three objectives. (1) Investigate the association between body mass index (BMI) and the efficacy of primary hip replacement using a patient-reported outcome measure (PROMs) with a measurement floor and ceiling, (2) Explore the performance of different estimation methods to estimate change in PROMs score following surgery using a simulation study and real word data where data has measurement floors and ceilings and (3) Lastly, develop guidance for practising researchers on the analysis of PROMs in the presence of floor and ceiling effects.

DESIGN

Simulation study and prospective national medical device register.

SETTING

National Register of Joint Replacement and Medical Devices.

METHODS

Using a Monte Carlo simulation study and data from a national joint replacement register (162 513 patients with pre- and post-surgery PROMs), we investigate simple approaches for the analysis of outcomes with floor and ceiling effects that are measured at two occasions: linear and Tobit regression (baseline adjusted analysis of covariance, change-score analysis, post-score analysis) in addition to linear and multilevel Tobit models.

PRIMARY OUTCOME

The primary outcome of interest is change in PROMs from pre-surgery to 6 months post-surgery.

RESULTS

Analysis of data with floor and ceiling effects with models that fail to account for these features induce substantial bias. Single-level Tobit models only correct for floor or ceiling effects when the exposure of interest is not associated with the baseline score. In observational data scenarios, only multilevel Tobit models are capable of providing unbiased inferences.

CONCLUSIONS

Inferences from pre- post-studies that fail to account for floor and ceiling effects may induce spurious associations with substantial risk of bias. Multilevel Tobit models indicate the efficacy of total hip replacement is independent of BMI. Restricting access to total hip replacement based on a patients BMI can not be supported by the data.

Comparison of three meta-analytic methods using data from digital interventions on type 2 diabetes

AIMS

Pooling the effect sizes of randomized controlled trials (RCTs) from continuous outcomes, such as glycated hemoglobin level (HbA1c), is an important method in evidence syntheses. However, due to challenges related to baseline imbalances and pre/post correlations, simple analysis of change scores (SACS) and simple analysis of final values (SAFV) meta-analyses result in under- or overestimation of effect estimates. This study was aimed to compare pooled effect sizes estimated by Analysis of Covariance (ANCOVA), SACS, and SAFV meta-analyses, using the example of RCTs of digital interventions with HbA1c as the main outcome.

MATERIALS AND METHODS

Three databases were systematically searched for RCTs published from 1993 through June 2017. Two reviewers independently assessed titles and abstracts using predefined eligibility criteria, assessed study quality, and extracted data, with disagreements resolved by arbitration from a third reviewer.

RESULTS

ANCOVA, SACS, and SAFV resulted in pooled HbA1c mean differences of -0.39% (95% CI: [-0.51, -0.26]), -0.39% (95% CI: [-0.51, -0.26]), and -0.34% (95% CI: [-0.48-0.19]), respectively. Removing studies with both high baseline imbalance (≥±0.2%) and pre/post correlation of ≥±0.6 resulted in a mean difference of -0.39% (95% CI: [-0.53, -0.26]), -0.40% (95% CI: [-0.54, -0.26]), and -0.33% (95% CI: [-0.48, -0.18]) with ANCOVA, SACS, and SAFV meta-analyses, respectively. Substantial heterogeneity was noted. Egger's test for funnel plot symmetry did not indicate evidence of publication bias for all methods.

CONCLUSION

By all meta-analytic methods, digital interventions appear effective in reducing HbA1c in type 2 diabetes. The effort to adjust for baseline imbalance and pre/post correlation using ANCOVA relies on the level of detail reported from individual studies. Reporting detailed summary data and, ideally, access to individual patient data of intervention trials are essential.

A Comparison of Methods for Estimating Relationships in the Change Between Two Time Points for Latent Variables

Collection and analysis of longitudinal data is an important tool in understanding growth and development over time in a whole range of human endeavors. Ideally, researchers working in the longitudinal framework are able to collect data at more than two points in time, as this will provide them with the potential for a deeper understanding of the development processes under study and a much broader array of statistical modeling options. However, in some circumstances data collection is limited to only two time points, perhaps because of resource limitations, issues with the context in which the data are collected, or the nature of the trait under study. In such instances, researchers may still want to learn about complex relationships in the data, such as the correlation between changes in latent traits that are being measured. However, with only two data points, standard approaches for modeling such relationships, such as growth curve modeling, cannot be used. The current simulation study compares the performance of two methods for estimating the correlations among changes in latent variables between two points in time, the two-wave latent change score model and the latent difference factor model. Results of the simulation study showed that both methods yielded generally accurate estimates of the correlation between changes in a latent trait, with relatively small standard errors. Estimation bias and standard errors were lower with larger samples, larger factor loading magnitudes, and more indicators per factor. Further comparisons between the methods and implications of these results are discussed.

Impact of analysing continuous outcomes using final values, change scores and analysis of covariance on the performance of meta-analytic methods: a simulation study

When meta-analysing intervention effects calculated from continuous outcomes, meta-analysts often encounter few trials, with potentially a small number of participants, and a variety of trial analytical methods. It is important to know how these factors affect the performance of inverse-variance fixed and DerSimonian and Laird random effects meta-analytical methods. We examined this performance using a simulation study. Meta-analysing estimates of intervention effect from final values, change scores, ANCOVA or a random mix of the three yielded unbiased estimates of pooled intervention effect. The impact of trial analytical method on the meta-analytic performance measures was important when there was no or little heterogeneity, but was of little relevance as heterogeneity increased. On the basis of larger than nominal type I error rates and poor coverage, the inverse-variance fixed effect method should not be used when there are few small trials. When there are few small trials, random effects meta-analysis is preferable to fixed effect meta-analysis. Meta-analytic estimates need to be cautiously interpreted; type I error rates will be larger than nominal, and confidence intervals will be too narrow. Use of trial analytical methods that are more efficient in these circumstances may have the unintended consequence of further exacerbating these issues. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.