1 year test–retest reliability of ImPACT in professional ice hockey players

Navigating uncertainty: exploring parents' knowledge of concussion management and neuropsychological baseline testing

Introduction

Parents play an important role in preventing and managing sport-related concussions among youth sport participants. Research indicates that parents understand the severity and consequences associated with the injury but gaps exist in their knowledge of its management. Neuropsychological baseline testing (NBT) is a modality that has gained interest in youth sport to purportedly better manage concussion injuries. Little is known about parents' perspectives on the use of NBT in the management process.

Methods

The present qualitative study used Protection Motivation Theory as a guiding framework and employed focus groups (N = 2) with parents (N = 11) to gain insight into parents' perceptions and experiences with concussion management, specifically focusing on NBT.

Results

Inductive Content Analysis developed a core theme of navigating uncertainty. Participants expressed uncertainty about the nature of concussion and its management process, where concussion was not always easy to identify, youth were not always reliable reporters, and there was no prescribed or proscribed path for recovery. Personal experience and concussion management policy provided participants with a degree of certainty in managing concussions. Participants gave NBT mixed reviews in potentially promoting greater certainty but also held reservations about its usefulness in concussion management.

Discussion

We discuss findings relative to existing knowledge and theory in youth sport concussion and identify implications for practice.

Psychometric Properties of Computerized Cognitive Tools and Standard Neuropsychological Tests Used to Assess Sport Concussion: A Systematic Review

Athletic programs are more frequently turning to computerized cognitive tools in order to increase efficiencies in concussion assessment. However, assessment using a traditional neuropsychological test battery may provide a more comprehensive and individualized evaluation. Our goal was to inform sport clinicians of the best practices for concussion assessment through a systematic literature review describing the psychometric properties of standard neuropsychological tests and computerized tools. We conducted our search in relevant databases including Ovid Medline, Web of Science, PsycINFO, and Scopus. Journal articles were included if they evaluated psychometric properties (e.g., reliability, sensitivity) of a cognitive assessment within pure athlete samples (up to 30 days post-injury). Searches yielded 4,758 unique results. Ultimately, 103 articles met inclusion criteria, all of which focused on adolescent or young adult participants. Test-retest reliability estimates ranged from .14 to .93 for computerized tools and .02 to .95 for standard neuropsychological tests, with strongest correlations on processing speed tasks for both modalities, although processing speed tasks were most susceptible to practice effects. Reliability was improved with a 2-factor model (processing speed and memory) and by aggregating multiple baseline exams, yet remained below acceptable limits for some studies. Sensitivity to decreased cognitive performance within 72 h of injury ranged from 45%-93% for computerized tools and 18%-80% for standard neuropsychological test batteries. The method for classifying cognitive decline (normative comparison, reliable change indices, regression-based methods) affected sensitivity estimates. Combining computerized tools and standard neuropsychological tests with the strongest psychometric performance provides the greatest value in clinical assessment. To this end, future studies should evaluate the efficacy of hybrid test batteries comprised of top-performing measures from both modalities.

Minimal Detectable Change for the ImPACT Test Administered Remotely

CONTEXT

Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is one of the most widely used computerized neurocognitive assessment batteries in athletics and serves as both a baseline and postinjury assessment. It has become increasingly popular to administer the ImPACT baseline test in an unsupervised remote environment; however, whether the lack of supervision affects the test-retest reliability is unknown.

OBJECTIVE

To establish the minimal detectable change (MDC) of composite scores from the ImPACT test when administered to National Collegiate Athletic Association Division I student-athletes in an unsupervised remote environment before 2 consecutive athletic seasons.

DESIGN

Cross-sectional study.

SETTING

Participants were provided with a unique link and detailed written instructions on how to complete the ImPACT test at home.

PATIENTS OR OTHER PARTICIPANTS

Division I student-athletes.

MAIN OUTCOME MEASURE(S)

Remote baseline ImPACT results from the 2020-2021 and 2021-2022 athletic seasons were analyzed. The MDC was calculated at the 95%, 90%, and 80% CIs for each of the ImPACT composite scores as well as the average and SD.

RESULTS

The MDC at the 95% CI was 18.6 for the verbal memory composite score, 24.44 for visual memory, 8.76 for visual motor, 0.14 for reaction time, and 6.13 for impulse control. One-way repeated-measures multivariate analysis of variance, repeated-measures analysis of variance, and Wilcoxon signed ranks tests suggested no difference in the composite scores and impulse control between time points.

CONCLUSIONS

The ImPACT composite scores and impulse control did not change between the 2 remote testing time points when administered approximately 1 year apart. Our study suggests that the MDC serves as a clinician's guide for evaluating changes in ImPACT baseline scores and in making clinical judgments on sport-related concussion when the test is administered at home.

The Concussion Challenge Assessment: Development and reliability of a novel gross motor assessment tool for paediatric concussion

Aims

The aim of this study was to develop a gross motor performance clinical assessment tool, the Concussion Challenge Assessment (CCA), for paediatric concussion populations.

Methods

An expert panel evaluated tasks from the Acquired Brain Injury Challenge Assessment to determine relevant tasks for a paediatric concussion population. These tasks were administered to a convenience sample of 854 healthy youth. An analysis of the response options for each task, considering task difficulty, was performed. The test-retest reliability of each task was considered to finalise the tool.

Results

The Acquired Brain Injury Challenge Assessment was reduced to six tasks (three coordination, two speed and agility, and one strength) to create the CCA. Population-specific 4-point response options were generated, which, upon examination of task difficulty, were revised as 5-point response sets to better capture performance differences. The test-retest reliability results led to acceptance of all six: three performance tasks and three exertion tasks.

Conclusion

This development of the CCA is an important step in creating a gross motor performance assessment tool that can assist in the determination of when youth are able to safely return to activity following a concussion.

Test-Retest Reliability of Remote ImPACT Administration

OBJECTIVE

To evaluate the performance and test-retest reliability obtained when administering a computerized baseline neurocognitive exam to NCAA Division I student-athletes in a controlled laboratory setting versus an uncontrolled remote location.

METHOD

A sample of 129 (female = 100) Division I student-athletes completed Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) pre-season assessments for two distinct and respective sports seasons in a controlled laboratory environment and an uncontrolled remote environment. Depending on the environment, participants were given verbal (controlled) or written (uncontrolled) guidelines for taking the test.

RESULTS

Multivariate repeated-measures ANOVA's determined that there were no within-subject differences between testing environments on ImPACT composite scores and cognitive efficiency index (CEI). The Chi-square test did not find any significant differences in impulse control or the number of invalid test scores, as determined by ImPACT, between environments. Intraclass correlations found the ImPACT subtest scores to range in test-retest reliability across testing environments, demonstrating moderate (verbal memory composite, r = 0.46; visual memory composite, r = 0.64; reaction time, r = 0.61; impulse control, r = 0.52; and CEI, r = 0.61) and good (visual motor composite, r = 0.77) test-retest reliability.

CONCLUSIONS

Results indicate that ImPACT is reliable between controlled and uncontrolled testing environments. This further suggests that ImPACT can be administered in a remote environment, pending specific adherence to testing instructions, or in the event of social distancing or isolation policies.

A much needed metric: Defining reliable and statistically meaningful change of the oral version Symbol Digit Modalities Test (SDMT)

BACKGROUND

The Symbol Digit Modalities Test (SDMT) has been recommended for use in clinical trials and outcome studies to monitor cognitive change. However, defining what is a meaningful change has been elusive for several years.

OBJECTIVE

The present investigation aimed to develop methods for assessing individual-level statistically significant change on the SDMT - reliable change indices (RCIs) and standardized regression-based (SRB) equations.

METHODS

A total of 219 healthy individuals completed the oral version SDMT at baseline, 6-month and 1-year follow-up.

RESULTS

The SDMT demonstrated high reliability across all time points (r's = 0.83 to 0.86). Reliable change scores of 7, 8, and 10 points for the 6-month intervals represented statistically meaningful change at the 0.70, 0.80, and 0.90 confidence intervals, respectively. Over 1-year, a difference of 8, 10, and 12 was statistically meaningful at the 0.70, 0.80, and 0.90 confidence intervals, respectively. SRB equations are also provided taking into account additional factors found to be predictive of SDMT scores over time.

CONCLUSION

Clinicians frequently denote a decline of 4 points on the SDMT as meaningful. Results in this large normative sample show that higher cut-points are needed to demonstrate statistically significant decline at the individual level. RCIs are provided for 6 month and one year assessment, which is typical in clinical practice and trials. SRB equations are also provided for use when applicable and may provide a more precise assessment of meaningful change.

The Neurological Consequences of Engaging in Australian Collision Sports

Collision sports are an integral part of Australian culture. The most common collision sports in Australia are Australian rules football, rugby union, and rugby league. Each of these sports often results in participants sustaining mild brain traumas, such as concussive and subconcussive injuries. However, the majority of previous studies and reviews pertaining to the neurological implications of sustaining mild brain traumas, while engaging in collision sports, have focused on those popular in North America and Europe. As part of this 2020 International Neurotrauma Symposium special issue, which highlights Australian neurotrauma research, this article will therefore review the burden of mild brain traumas in Australian collision sports athletes. Specifically, this review will first provide an overview of the consequences of mild brain trauma in Australian collision sports, followed by a summary of the previous studies that have investigated neurocognition, ocular motor function, neuroimaging, and fluid biomarkers, as well as neuropathological outcomes in Australian collision sports athletes. A review of the literature indicates that although Australians have contributed to the field, several knowledge gaps and limitations currently exist. These include important questions related to sex differences, the identification and implementation of blood and imaging biomarkers, the need for consistent study designs and common data elements, as well as more multi-modal studies. We conclude that although Australia has had an active history of investigating the neurological impact of collision sports participation, further research is clearly needed to better understand these consequences in Australian athletes and how they can be mitigated.

Post-Concussion Acute Signs and Reliable Cognitive Decline in a Finnish Youth Ice Hockey Sample

OBJECTIVE

In sports concussion research, the importance of an individualized approach incorporating neuropsychological assessment data has been emphasized. This study examined the impact of acute signs of concussion on post-injury cognitive functioning using reliable change methodology in a sample of Finnish, elite-level, youth ice hockey players.

METHODS

From a sample of 1,823 players (all male, 14-20 years old) who completed preseason baseline testing with the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT®) battery, two subgroups were identified. First, in total, 312 uninjured athletes, who completed baseline testing twice-1 year apart. The scores were contrasted to calculate reliable change indices (RCIs). Second, from a subsample of 570 athletes participating in an intensive follow-up arm of the project, the analysis included 32 concussed athletes. The RCIs were determined for the five ImPACT composite scores and used in identifying athletes with declined performance 3 days post-injury.

RESULTS

Test-retest reliability ranged from .39 to .71. Athletes who had experienced an acute loss of consciousness, amnesia, or postural instability had increased odds for declines in two or more areas assessed by ImPACT (odds ratio = 7.67-8.00, p < .05). In contrast, acute disorientation or vacant look did not lead to cognitive change that met the reliable change threshold.

CONCLUSIONS

The reliability coefficients and RCIs differed from those published earlier emphasizing the importance of national reference values. The presence of acute loss of consciousness, amnesia, or postural instability may indicate a more severe injury and predict the need for more intensive cognitive follow-up.

A quick assessment of reliable change in fatigue: Reliable change indices of the modified fatigue impact scale - 5 item (MFIS-5)

BACKGROUND

Reliably monitoring changes in fatigue is an ongoing concern.

OBJECTIVE

Evaluate reliable change using the Modified Fatigue Impact Scale 5-item version (MFIS-5) in people with MS (PwMS).

METHODS

The MFIS-5 was administered at three time points in 157 PwMS. Test-retest reliability and reliable change scores were calculated at the 0.70, 0.80, 0.90, and 0.95 confidence intervals.

RESULTS

Difference scores of 3, 4, 5, and 6 represent statistically meaningful change at the 0.70, 0.80, 0.90, and 0.95 confidence intervals, respectively.

CONCLUSION

Cut points derived from this study and prior work can help reliably assess changes in fatigue over time.

Sensitivity and Specificity of Computer-Based Neurocognitive Tests in Sport-Related Concussion: Findings from the NCAA-DoD CARE Consortium

BACKGROUND

To optimally care for concussed individuals, a multi-dimensional approach is critical and a key component of this assessment in the athletic environment is computer-based neurocognitive testing. However, there continues to be concerns about the reliability and validity of these testing tools. The purpose of this study was to determine the sensitivity and specificity of three common computer-based neurocognitive tests (Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT], CNS Vital Signs, and CogState Computerized Assessment Tool [CCAT]), to provide guidance on their clinical utility.

METHODS

This study analyzed assessments from a cohort of collegiate athletes and non-varsity cadets from the NCAA-DoD CARE Consortium. The data were collected from 2014-2018. Study participants were divided into two testing groups [concussed, n = 1414 (baseline/24-48 h) and healthy, n = 8305 (baseline/baseline)]. For each test type, change scores were calculated for the components of interest. Then, the Normative Change method, which used normative data published in a similar cohort, and the Reliable Change Index (RCI) method were used to determine if the change scores were significant.

RESULTS

Using the Normative Change method, ImPACT performed best with an 87.5%-confidence interval and 1 number of components failed (NCF; sensitivity = 0.583, specificity = 0.625, F1 = 0.308). CNS Vital Signs performed best with a 90%-confidence interval and 1 NCF (sensitivity = 0.587, specificity = 0.532, F1 = 0.314). CCAT performed best when using a 75%-confidence interval and 2 NCF (sensitivity = 0.513, specificity = 0.715, F1 = 0.290). When using the RCI method, ImPACT performed best with an 87.5%-confidence interval and 1 NCF (sensitivity = 0.626, specificity = 0.559, F1 = 0.297).

CONCLUSION

When considering all three computer-based neurocognitive tests, the overall low sensitivity and specificity results provide additional evidence for the use of a multi-dimensional assessment for concussion diagnosis, including symptom evaluation, postural control assessment, neuropsychological status, and other functional assessments.

Estimated Age of First Exposure to American Football and Neurocognitive Performance Amongst NCAA Male Student-Athletes: A Cohort Study

BACKGROUND

Repetitive head impacts in young athletes are potentially detrimental to later life (e.g., age 50 + years) neurological function; however, it is unknown what the short-term effects (e.g., age 20 years) are in collegiate student-athletes.

OBJECTIVE

The purpose of this study was to determine the effect of the estimated age of first exposure to American tackle football participation on neurocognitive performance and symptom severity scores in collegiate student-athletes.

METHODS

We used a cohort study in which neurocognitive performance was assessed using the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) test in 4376 male athletes (age 19.3 ± 1.5 years, mass 96.3 ± 20.3 kg, height 185.0 ± 7.4 cm). Athletes were grouped by sport participation [American football (n = 3462) or non-contact (n = 914)] and estimated age of first exposure [< 12 years (n = 3022) or ≥ 12 years (n = 1354)]. The outcome measures were the four primary cognitive scores and the symptom severity score from ImPACT. We assessed primary outcomes across groups, controlling for age, learning accommodations, and concussion history.

RESULTS

Neurocognitive performance was not associated with the estimated age of first exposure-by-group interaction.

CONCLUSION

Our findings indicate that participation in American tackle football before age 12 years does not result in neurocognitive deficits in college. Therefore, we suggest the following: the consequences of early exposure to repetitive head impacts do not manifest by college, the ImPACT test was not sensitive enough to identify the effects of an earlier estimated age of first exposure, or there is no association between an earlier estimated age of first exposure and neurocognitive functioning. Future longitudinal studies are warranted.

Testing the hybrid battery approach to evaluating sports-related concussion in the National Hockey League: A factor analytic study

Objective: There has been an increased interest in the use of "Hybrid" neuropsychological test batteries to evaluate neurocognitive functioning prior to and following sports-related concussion. Typically, these test batteries include a combination of traditional "paper and pencil" (P&P) and computerized measures. The National Hockey League (NHL) employs a Hybrid approach consisting of ImPACT and a brief battery of P&P measures. The purpose of this paper was to examine the NHL's Hybrid model by means of factor analytic techniques in order to determine the extent to which the measures included in the battery are independent of each other or share common variance.Method: Principal components analyses with promax rotations were conducted on the P&P measures alone and when combined with ImPACT. A total of 360 recently concussed NHL players underwent testing using the combined hybrid battery.Results: A principal components analysis with promax rotation of the combined hybrid battery yielded 5 factors (Verbal Learning/Memory, Visual Learning/Memory, Processing Speed/Executive Functioning, Cued/Recognition Memory and Reaction Time/Speed). The factors appear to be relatively independent of each other with the P&P measures comprising the Verbal Learning/Memory, Visual Learning/Memory, and Processing Speed/Executive Functioning and ImPACT loading on the Cued/Recognition Memory and Reaction Time/Speed factors.Conclusions: These data provide support for the Hybrid model with evidence that the P&P measures and ImPACT do not measure the same domains of cognitive functioning. The clinical implications of the findings are discussed in detail.

Validation of a Concussion Screening Battery for Use in Medical Settings: Predicting Centers for Disease Control Concussion Symptoms in Children and Adolescents

Objective

Effective screening for concussion is increasingly important, and medical professionals play a critical role in diagnostic and return-to-play decisions. However, few well-validated measures are available to assist in those decisions. This study aims to determine whether previously validated measures assessing neurocognitive and neurobehavioral abilities can predict Centers for Disease Control (CDC) concussion symptom endorsement in a sample of child or youth athletes.

Method

Participants were 113 individuals, aged 6–17, representing 29 consecutive cases undergoing a post-concussion evaluation by a pediatric neurologist and 84 consecutive cases completing standardized baseline assessments (i.e., not being evaluated as a follow-up to a concussion). All participants completed the same standardized battery of tests comprised of the Connors’ Continuous Performance Test (CPT 3), the Balance Error Scoring System (BESS), and the NIH 4-Meter Gait Test as well as completing a checklist of CDC concussion symptoms.

Results

Regression analyses indicate that the screening battery explained 33% of the variance (d = 1.4) in concussion symptom endorsement, after controlling for age. The neurocognitive test alone (CPT 3) accounts for 21.5% of the variance (d = 1.05) in symptoms after controlling for age, and the neurobehavioral measures (BESS and NIH 4-Meter Gait) then account for an additional 11.5% variance (accounting for 18.6% variance, d = .96, when entered first). These effect sizes are considered large to very large and reflect a marked increase in predictive validity relative to existing measures commonly used in concussion assessments.

Conclusions

A relatively brief screening battery can function in medical settings to predict significant and substantial variability in CDC concussion symptoms in a pediatric sample.

Prevalence of Invalid Performance on Baseline Testing for Sport-Related Concussion by Age and Validity Indicator

Importance

Estimated base rates of invalid performance on baseline testing (base rates of failure) for the management of sport-related concussion range from 6.1% to 40.0%, depending on the validity indicator used. The instability of this key measure represents a challenge in the clinical interpretation of test results that could undermine the utility of baseline testing.

Objectives

To determine the prevalence of invalid performance on baseline testing and to assess whether the prevalence varies as a function of age and validity indicator.

Design, Setting, and Participants

This retrospective, cross-sectional study included data collected between January 1, 2012, and December 31, 2016, from a clinical referral center in the Midwestern United States. Participants included 7897 consecutively tested, equivalently proportioned male and female athletes aged 10 to 21 years, who completed baseline neurocognitive testing for the purpose of concussion management.

Interventions

Baseline assessment was conducted with the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), a computerized neurocognitive test designed for assessment of concussion.

Main Outcomes and Measures

Base rates of failure on published ImPACT validity indicators were compared within and across age groups. Hypotheses were developed after data collection but prior to analyses.

Results

Of the 7897 study participants, 4086 (51.7%) were male, mean (SD) age was 14.71 (1.78) years, 7820 (99.0%) were primarily English speaking, and the mean (SD) educational level was 8.79 (1.68) years. The base rate of failure ranged from 6.4% to 47.6% across individual indicators. Most of the sample (55.7%) failed at least 1 of 4 validity indicators. The base rate of failure varied considerably across age groups (117 of 140 [83.6%] for those aged 10 years to 14 of 48 [29.2%] for those aged 21 years), representing a risk ratio of 2.86 (95% CI, 2.60-3.16; P < .001).

Conclusions and Relevance

The results for base rate of failure were surprisingly high overall and varied widely depending on the specific validity indicator and the age of the examinee. The strong age association, with 3 of 4 participants aged 10 to 12 years failing validity indicators, suggests that the clinical interpretation and utility of baseline testing in this age group is questionable. These findings underscore the need for close scrutiny of performance validity indicators on baseline testing across age groups.

Preliminary normative study of ImPACT® in Finnish professional male ice hockey players

Assessment of cognition is an important part of concussion management. The common paradigm of baseline and postinjury evaluations is recommended but due to the often lacking baseline data, reliable normative values are needed. The Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT®) battery is a widely used method of cognitive assessment offering several language options. There are few comparative studies between language groups on this test battery. ImPACT was administered at baseline to 184 Finnish male Professional Ice Hockey league players. The performance of the Finnish sample differed from previously published English and Czech language samples on the Visual Motor Speed and Reaction Time composites with medium effect-sizes (d = 0.38-0.52). Age, but not education or prior concussions, was associated with ImPACT performance. ImPACT performance is not uniform across language and culture groups and the findings highlight the need for language-specific norms. Finnish reference values for ImPACT for ages <20, 20-26, and >26 are presented. A similar approach is encouraged for other languages.

Signs of cognitive dysfunction in adult players of club level rugby

The objective of this study was to investigate players of club level Rugby Union (hereafter rugby) over one rugby season, for signs of residual cognitive dysfunction due to repeated concussive events incurred during long-term participation in the sport. Adult male players of club level rugby, without a diagnosis of a concussion during the season ( n = 20), were compared with non-contact club level sports participants ( n = 22) of equivalent sex, age, education, and estimated IQ at pre-, mid-, and post-season intervals. Measures included the ImPACT Verbal Memory, Visual Memory, Visual Motor Speed and Reaction Time composites, and the Purdue Pegboard test. Statistical group comparisons revealed consistently poorer performance for rugby players compared with controls for ImPACT Visual Motor Speed and Reaction Time composites at all test intervals, and for the Purdue Pegboard tasks at the first two test intervals. Repeat measures comparisons across the three test intervals demonstrated differential learning patterns between groups on ImPACT Visual Memory, ImPACT Reaction Time, and Purdue Both tasks, suggestive of cognitive vulnerability in the rugby group. Overall, the results reveal deleterious cognitive performance in adult club level rugby players relative to equivalent non-contact sports controls. The finding serves to endorse indications from other studies that demonstrate persistent brain injury effects in association with long-term participation in a contact sport. Recommendations for further research and management of concussion in rugby players are discussed.

Reliability of Computerized Neurocognitive Tests for Concussion Assessment: A Meta-Analysis

OBJECTIVE

  Although widely used, computerized neurocognitive tests (CNTs) have been criticized because of low reliability and poor sensitivity. A systematic review was published summarizing the reliability of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) scores; however, this was limited to a single CNT. Expansion of the previous review to include additional CNTs and a meta-analysis is needed. Therefore, our purpose was to analyze reliability data for CNTs using meta-analysis and examine moderating factors that may influence reliability.

DATA SOURCES

  A systematic literature search (key terms: reliability, computerized neurocognitive test, concussion) of electronic databases (MEDLINE, PubMed, Google Scholar, and SPORTDiscus) was conducted to identify relevant studies.

STUDY SELECTION

  Studies were included if they met all of the following criteria: used a test-retest design, involved at least 1 CNT, provided sufficient statistical data to allow for effect-size calculation, and were published in English.

DATA EXTRACTION

  Two independent reviewers investigated each article to assess inclusion criteria. Eighteen studies involving 2674 participants were retained. Intraclass correlation coefficients were extracted to calculate effect sizes and determine overall reliability. The Fisher Z transformation adjusted for sampling error associated with averaging correlations. Moderator analyses were conducted to evaluate the effects of the length of the test-retest interval, intraclass correlation coefficient model selection, participant demographics, and study design on reliability. Heterogeneity was evaluated using the Cochran Q statistic.

DATA SYNTHESIS

  The proportion of acceptable outcomes was greatest for the Axon Sports CogState Test (75%) and lowest for the ImPACT (25%). Moderator analyses indicated that the type of intraclass correlation coefficient model used significantly influenced effect-size estimates, accounting for 17% of the variation in reliability.

CONCLUSIONS

  The Axon Sports CogState Test, which has a higher proportion of acceptable outcomes and shorter test duration relative to other CNTs, may be a reliable option; however, future studies are needed to compare the diagnostic accuracy of these instruments.

Evaluating the test-retest reliability of symptom indices associated with the ImPACT post-concussion symptom scale (PCSS)

OBJECTIVE

The Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) is a commonly used tool in sports concussion assessment. While test-retest reliabilities have been established for the ImPACT cognitive composites, few studies have evaluated the psychometric properties of the ImPACT's Post-Concussion Symptom Scale (PCSS). The purpose of this study was to establish the test-retest reliability of symptom indices associated with the PCSS.

PARTICIPANTS AND METHODS

Participants included 38 undergraduate students (50.0% male) who underwent neuropsychological testing as part of their participation in their psychology department's research subject pool. The majority of the participants were Caucasian (94.7%) and had no history of concussion (73.7%). All participants completed the ImPACT at two time points, approximately 6 weeks apart. The PCSS was the main outcome measure, and eight symptom indices were calculated (a total symptom score, three symptom summary indices, and four symptom clusters).

RESULTS

Pearson correlations (r) and intraclass correlation coefficients (ICCs) were computed as measures of test-retest reliability. Overall, reliabilities ranged from low to high (r = .44 to .80; ICC = .44 to .77). The cognitive symptom cluster exhibited the highest test-retest reliability (r = .80, ICC = .77), followed by the positive symptom total (PST) index, an indicator of the total number of symptoms endorsed (r = .71, ICC = .69). In contrast, the commonly used total symptom score showed lower test-retest reliability (r = .67, ICC = .62). Paired-samples t tests revealed no significant differences between test and retest for any of the symptom variables (all p > .01). Finally, reliable change indices (RCI) were computed to determine whether differences observed between test and retest represented clinically significant change. RCI values were provided for each symptom index at the 80%, 90%, and 95% confidence intervals.

CONCLUSION

These results suggest that evaluating additional symptom indices beyond the total symptom score from the PCSS is beneficial. Findings from this study can be applied to athlete samples to assess reliable change in symptoms following concussion.

Measuring cognitive change with ImPACT: the aggregate baseline approach

OBJECTIVE

The Immediate Post-Concussion Assessment and Cognitive Test (ImPACT) is commonly used to assess baseline and post-injury cognition among athletes in North America. Despite this, several studies have questioned the reliability of ImPACT when given at intervals employed in clinical practice. Poor test-retest reliability reduces test sensitivity to cognitive decline, increasing the likelihood that concussed athletes will be returned to play prematurely. We recently showed that the reliability of ImPACT can be increased when using a new composite structure and the aggregate of two baselines to predict subsequent performance. The purpose of the present study was to confirm our previous findings and determine whether the addition of a third baseline would further increase the test-retest reliability of ImPACT.

METHODS

Data from 97 English speaking professional hockey players who had received at least 4 ImPACT baseline evaluations were extracted from a National Hockey League Concussion Program database. Linear regression was used to determine whether each of the first three testing sessions accounted for unique variance in the fourth testing session.

RESULTS

Results confirmed that the aggregate baseline approach improves the psychometric properties of ImPACT, with most indices demonstrating adequate or better test-retest reliability for clinical use.

CONCLUSIONS

The aggregate baseline approach provides a modest clinical benefit when recent baselines are available - and a more substantial benefit when compared to approaches that obtain baseline measures only once during the course of a multi-year playing career. Pending confirmation in diverse samples, neuropsychologists are encouraged to use the aggregate baseline approach to best quantify cognitive change following sports concussion.

Normative Data for a Comprehensive Neuropsychological Test Battery used in the Assessment of Sports-Related Concussion

Objective

The use of normative data is a hallmark of the neuropsychological assessment process. Within the context of sports-related concussion, utilizing normative data is especially essential when individualized baseline data are unavailable for comparison. The primary purpose of this study was to establish normative data for a comprehensive neuropsychological test battery used in the assessment of sports-related concussion. A secondary aim was to provide normative data for pertinent demographic variables relevant to the assessment of college athletes, including sex, previous head injuries (PHI), and history of attention deficit hyperactivity disorder (ADHD)/learning disability (LD).

Method

Participants included male and female college athletes (N = 794) who were involved in a concussion management program at an NCAA Division I university between 2002 and 2015. Athletes were administered a comprehensive neuropsychological test battery at baseline designed to assess the following cognitive domains: learning and memory, attention and concentration, processing speed, and executive functioning. The test battery primarily comprises paper-and-pencil measures.

Results

Normative data are presented for the overall athlete sample. Additional sub-norms are then provided for specified demographic populations (i.e., sex, PHIs, and history of ADHD/LD). Findings indicate that there are mild cognitive differences between men and women, as well as between those athletes with and without a history of ADHD/LD. Given these findings, additional norms are provided for men and women with and without a history of ADHD/LD.

Conclusions

In the absence of baseline testing, the normative data presented here can be used clinically to assess athletes' cognitive functioning post-concussion.

Examining Recovery Trajectories After Sport-Related Concussion With a Multimodal Clinical Assessment Approach

BACKGROUND

Previous research estimates that the majority of athletes with sport-related concussion (SRC) will recover between 7 and 10 days after injury. This short temporal window of recovery is based predominately on symptom resolution and cognitive improvement and does not accurately reflect recent advances in the clinical assessment model.

OBJECTIVE

To characterize SRC recovery at 1-week postinjury time intervals on symptom, neurocognitive, and vestibular-oculomotor outcomes and to examine sex differences in SRC recovery time.

METHODS

A prospective, repeated-measures design was used to examine the temporal resolution of neurocognitive, symptom, and vestibular-oculomotor impairment in 66 subjects (age, 16.5 ± 1.9 years; range, 14-23 years; 64% male) with SRC.

RESULTS

Recovery time across all outcomes was between 21 and 28 days after SRC for most athletes. Symptoms demonstrated the greatest improvement in the first 2 weeks, although neurocognitive impairment lingered across various domains up to 28 days after SRC. Vestibular-oculomotor decrements also resolved between 1 and 3 weeks after injury. There were no sex differences in neurocognitive recovery. Male subjects were more likely to be asymptomatic by the fourth week and reported less vestibular-oculomotor impairment than female subjects at weeks 1 and 2.

CONCLUSION

When the recommended "comprehensive" approach is used for concussion assessment, recovery time for SRC is approximately 3 to 4 weeks, which is longer than the commonly reported 7 to 14 days. Sports medicine clinicians should use a variety of complementing assessment tools to capture the heterogeneity of SRC.

The influence of validity criteria on Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) test-retest reliability among high school athletes

OBJECTIVE

Research findings to date on the stability of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) Composite scores have been inconsistent, requiring further investigation. The use of test validity criteria across these studies also has been inconsistent. Using multiple measures of stability, we examined test-retest reliability of repeated ImPACT baseline assessments in high school athletes across various validity criteria reported in previous studies.

METHOD

A total of 1146 high school athletes completed baseline cognitive testing using the online ImPACT test battery at two time periods of approximately two-year intervals. No participant sustained a concussion between assessments. Five forms of validity criteria used in previous test-retest studies were applied to the data, and differences in reliability were compared.

RESULTS

Intraclass correlation coefficients (ICCs) ranged in composite scores from .47 (95% confidence interval, CI [.38, .54]) to .83 (95% CI [.81, .85]) and showed little change across a two-year interval for all five sets of validity criteria. Regression based methods (RBMs) examining the test-retest stability demonstrated a lack of significant change in composite scores across the two-year interval for all forms of validity criteria, with no cases falling outside the expected range of 90% confidence intervals.

CONCLUSION

The application of more stringent validity criteria does not alter test-retest reliability, nor does it account for some of the variation observed across previously performed studies. As such, use of the ImPACT manual validity criteria should be utilized in the determination of test validity and in the individualized approach to concussion management. Potential future efforts to improve test-retest reliability are discussed.

Long-term Stability and Reliability of Baseline Cognitive Assessments in High School Athletes Using ImPACT at 1-, 2-, and 3-year Test-Retest Intervals

OBJECTIVE

The ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing) neurocognitive testing battery is a widely used tool used for the assessment and management of sports-related concussion. Research on the stability of ImPACT in high school athletes at a 1- and 2-year intervals have been inconsistent, requiring further investigation. We documented 1-, 2-, and 3-year test-retest reliability of repeated ImPACT baseline assessments in a sample of high school athletes, using multiple statistical methods for examining stability.

METHODS

A total of 1,510 high school athletes completed baseline cognitive testing using online ImPACT test battery at three time periods of approximately 1- (N = 250), 2- (N = 1146), and 3-year (N = 114) intervals. No participant sustained a concussion between assessments.

RESULTS

Intraclass correlation coefficients (ICCs) ranged in composite scores from 0.36 to 0.90 and showed little change as intervals between assessments increased. Reliable change indices and regression-based measures (RBMs) examining the test-retest stability demonstrated a lack of significant change in composite scores across the various time intervals, with very few cases (0%-6%) falling outside of 95% confidence intervals.

CONCLUSION

The results suggest ImPACT composites scores remain considerably stability across 1-, 2-, and 3-year test-retest intervals in high school athletes, when considering both ICCs and RBM. Annually ascertaining baseline scores continues to be optimal for ensuring accurate and individualized management of injury for concussed athletes. For instances in which more recent baselines are not available (1-2 years), clinicians should seek to utilize more conservative range estimates in determining the presence of clinically meaningful change in cognitive performance.

Participation in Pre-High School Football and Neurological, Neuroradiological, and Neuropsychological Findings in Later Life: A Study of 45 Retired National Football League Players

BACKGROUND

A recent study found that an earlier age of first exposure (AFE) to tackle football was associated with long-term neurocognitive impairment in retired National Football League (NFL) players.

PURPOSE

To assess the association between years of exposure to pre-high school football (PreYOE) and neuroradiological, neurological, and neuropsychological outcome measures in a different sample of retired NFL players.

STUDY DESIGN

Cross-sectional study; Level of evidence, 3.

METHODS

Forty-five former NFL players were included in this study. All participants prospectively completed extensive history taking, a neurological examination, brain magnetic resonance imaging, and a comprehensive battery of neuropsychological tests. To measure the associations between PreYOE and these outcome measures, multiple regression models were utilized while controlling for several covariates.

RESULTS

After applying a Bonferroni correction for multiple comparisons, none of the neurological, neuroradiological, or neuropsychological outcome measures yielded a significant relationship with PreYOE. A second Bonferroni-corrected analysis of a subset of these athletes with self-reported learning disability yielded no significant relationships on paper-and-pencil neurocognitive tests but did result in a significant association between learning disability and computerized indices of visual motor speed and reaction time.

CONCLUSION

The current study failed to replicate the results of a prior study, which concluded that an earlier AFE to tackle football might result in long-term neurocognitive deficits. In 45 retired NFL athletes, there were no associations between PreYOE and neuroradiological, neurological, and neuropsychological outcome measures.

Test-Retest Reliability of Computerized Neurocognitive Testing in Youth Ice Hockey Players

Computerized neurocognitive tests are frequently used to assess pediatric sport-related concussions; however, only 1 study has focused on the test-retest reliability of the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) in high school athletes and age influences have largely been ignored. Therefore, the purpose was to investigate the test-retest reliability of ImPACT and underlying age influences in a pediatric population. Two hundred (169 men and 31 women) youth ice hockey players completed ImPACT before/after a 6-month season. Reliability was assessed using Pearson correlation coefficients, intraclass correlation coefficients (ICCs), and regression-based methods (RBz). ICCs for the sample ranged from .48 to .75 (single)/.65 to .86 (average). In general, the older athletes (15-18: Single/Average ICCs = .35-.75/.52-.86) demonstrated greater reliability across composites than the younger athletes (11-14: Single/Average ICCs = .54-.63/.70-.77). Although there was variation in athletes' performance across two test administrations, RBz revealed that only a small percentage of athletes performed beyond 80%, 90%, and 95% confidence intervals. Statistical metrics demonstrated reliability coefficients for ImPACT composites in a pediatric sample similar to previous studies, and also revealed important age-related influences.

Utilization of the clinical laboratory for the implementation of concussion biomarkers in collegiate football and the necessity of personalized and predictive athlete specific reference intervals

Background

A continued interest in concussion biomarkers makes the eventual implementation of identified biomarkers into routine concussion assessment an eventual reality. We sought to develop and test an interdisciplinary approach that could be used to integrate blood-based biomarkers into the established concussion management program for a collegiate football team.

Methods

We used a CLIA-certified laboratory for all testing and chose biomarkers where clinically validated testing was available as would be required for results used in clinical decision making. We summarized the existing methods and results for concussion assessment across an entire season to identify and demonstrate the challenges with the eventual integration of a parallel process using blood-based tests for concussion management. We analyzed the results of the biomarkers chosen for trends consistent with the outcome assessments provided from the current concussion management protocols.

Results

Baseline samples were collected with three additional post-concussion samples collected at three separate time points from players with a diagnosed concussion (n = 12). A summary of results from currently used concussion assessment tools were compared to the representative biomarkers S100B and NSE results. Nine sport-related concussions occurred during practice and three during play. For S100B, 50 % had follow-up testing results lower than the post-injury result. In contrast, 92 % of NSE follow-up results were lower than post-injury. One hundred percent of the results for S100B and NSE were within the athlete-derived reference intervals upon return-to-play and season end.

Conclusions

The reported workflow provides a framework for the eventual implementation of biomarkers for concussion assessment into existing assessment protocols and strengthens the need for reliance on clinical laboratory testing. Athlete-specific reference intervals will be required to adequately interpret results.

Electronic supplementary material

The online version of this article (doi:10.1186/s13167-016-0050-x) contains supplementary material, which is available to authorized users.

Two baselines are better than one: Improving the reliability of computerized testing in sports neuropsychology

Computerized neuropsychological tests are frequently used to assist in return-to-play decisions following sports concussion. However, due to concerns about test reliability, the Centers for Disease Control and Prevention recommends yearly baseline testing. The standard practice that has developed in baseline/postinjury comparisons is to examine the difference between the most recent baseline test and postconcussion performance. Drawing from classical test theory, the present study investigated whether temporal stability could be improved by taking an alternate approach that uses the aggregate of 2 baselines to more accurately estimate baseline cognitive ability. One hundred fifteen English-speaking professional hockey players with 3 consecutive Immediate Postconcussion Assessment and Testing (ImPACT) baseline tests were extracted from a clinical program evaluation database overseen by the National Hockey League and National Hockey League Players' Association. The temporal stability of ImPACT composite scores was significantly increased by aggregating test performance during Sessions 1 and 2 to predict performance during Session 3. Using this approach, the 2-factor Memory (r = .72) and Speed (r = .79) composites of ImPACT showed acceptable long-term reliability. Using the aggregate of 2 baseline scores significantly improves temporal stability and allows for more accurate predictions of cognitive change following concussion. Clinicians are encouraged to estimate baseline abilities by taking into account all of an athlete's previous baseline scores.

Prospective, Head-to-Head Study of Three Computerized Neurocognitive Assessment Tools (CNTs): Reliability and Validity for the Assessment of Sport-Related Concussion

Limited data exist comparing the performance of computerized neurocognitive tests (CNTs) for assessing sport-related concussion. We evaluated the reliability and validity of three CNTs-ANAM, Axon Sports/Cogstate Sport, and ImPACT-in a common sample. High school and collegiate athletes completed two CNTs each at baseline. Concussed (n=165) and matched non-injured control (n=166) subjects repeated testing within 24 hr and at 8, 15, and 45 days post-injury. Roughly a quarter of each CNT's indices had stability coefficients (M=198 day interval) over .70. Group differences in performance were mostly moderate to large at 24 hr and small by day 8. The sensitivity of reliable change indices (RCIs) was best at 24 hr (67.8%, 60.3%, and 47.6% with one or more significant RCIs for ImPACT, Axon, and ANAM, respectively) but diminished to near the false positive rates thereafter. Across time, the CNTs' sensitivities were highest in those athletes who became asymptomatic within 1 day before neurocognitive testing but was similar to the tests' false positive rates when including athletes who became asymptomatic several days earlier. Test-retest reliability was similar among these three CNTs and below optimal standards for clinical use on many subtests. Analyses of group effect sizes, discrimination, and sensitivity and specificity suggested that the CNTs may add incrementally (beyond symptom scores) to the identification of clinical impairment within 24 hr of injury or within a short time period after symptom resolution but do not add significant value over symptom assessment later. The rapid clinical recovery course from concussion and modest stability probably jointly contribute to limited signal detection capabilities of neurocognitive tests outside a brief post-injury window. (JINS, 2016, 22, 24-37).

Two-year Test-Retest Reliability of ImPACT in High School Athletes

This research evaluated the 2-year test-retest reliability of the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) neuropsychological battery, and clarified the need for biennial updated baseline testing of high school athletes. This study compared the baseline test scores of 212 non-concussed athletes that were obtained in Grade 9 and again 2 years later when they were in Grade 11. Regression-based methods indicated that 4 of the 5 ImPACT scores were stable over 2 years, as they fell within the 80% and 95% confidence intervals (CIs). The results suggested that updating baseline testing for high school athletes after 2 years is not necessary. Further research into the consistency of computerized neuropsychological tests over 2 years with high school athletes is recommended.

Multivariate analysis of traumatic brain injury: development of an assessment score

Important challenges for the diagnosis and monitoring of mild traumatic brain injury (mTBI) include the development of plasma biomarkers for assessing neurologic injury, monitoring pathogenesis, and predicting vulnerability for the development of untoward neurologic outcomes. While several biomarker proteins have shown promise in this regard, used individually, these candidates lack adequate sensitivity and/or specificity for making a definitive diagnosis or identifying those at risk of subsequent pathology. The objective for this study was to evaluate a panel of six recognized and novel biomarker candidates for the assessment of TBI in adult patients. The biomarkers studied were selected on the basis of their relative brain-specificities and potentials to reflect distinct features of TBI mechanisms including (1) neuronal damage assessed by neuron-specific enolase (NSE) and brain derived neurotrophic factor (BDNF); (2) oxidative stress assessed by peroxiredoxin 6 (PRDX6); (3) glial damage and gliosis assessed by glial fibrillary acidic protein and S100 calcium binding protein beta (S100b); (4) immune activation assessed by monocyte chemoattractant protein 1/chemokine (C–C motif) ligand 2 (MCP1/CCL2); and (5) disruption of the intercellular adhesion apparatus assessed by intercellular adhesion protein-5 (ICAM-5). The combined fold-changes in plasma levels of PRDX6, S100b, MCP1, NSE, and BDNF resulted in the formulation of a TBI assessment score that identified mTBI with a receiver operating characteristic (ROC) area under the curve of 0.97, when compared to healthy controls. This research demonstrates that a profile of biomarker responses can be used to formulate a diagnostic score that is sensitive for the detection of mTBI. Ideally, this multivariate assessment strategy will be refined with additional biomarkers that can effectively assess the spectrum of TBI and identify those at particular risk for developing neuropathologies as consequence of a mTBI event.