Test-Retest Reliability of Concussion Baseline Assessments in United States Service Academy Cadets: A Report from the National Collegiate Athletic Association (NCAA)-Department of Defense (DoD) CARE Consortium

Female Collegiate Athletes' Concussion Characteristics and Recovery Patterns: A Report from the NCAA-DoD CARE Consortium

Concussion has been described in the United States (US) collegiate student-athlete population, but female-specific findings are often underrepresented and underreported. Our study aimed to describe female collegiate student-athletes' initial injury characteristics and return to activity outcomes following concussion. Female collegiate student-athletes (n = 1393) from 30-US institutions experienced a concussion and completed standardized, multimodal concussion assessments from pre-injury through unrestricted return to play (uRTP) in this prospective, longitudinal cohort study. Initial injury presentation characteristics, assessment, and return to activity outcomes [<48-h (acute), return to learn, initiate return to play (iRTP), uRTP] were collected. We used descriptive statistics to report injury characteristics, return to activity outcomes, and post-injury assessment performance change categorization (worsened, unchanged, improved) based on change score confidence rank criteria across sport contact classifications [contact (n = 661), limited (n = 446), non-contact (n = 286)]. The median (25th to 75th percentile) days to return to learn was 6.0 (3.0-10.0), iRTP was 8.1 (4.8-13.8), and uRTP was 14.8 (9.9-24.0), but varied by contact classification. Across contact levels, the majority experienced worse SCAT total symptom severity (72.8-82.6%), ImPACT reaction time (91.2-92.6%), and BSI-18 total score (45.2-51.8%) acutely relative to baseline, but unchanged BESS total errors (58.0-60.9%), SAC total score (71.5-76.1%), and remaining ImPACT domains (50.6-66.5%). Our findings provide robust estimates of the typical female collegiate student-athlete presentation and recovery trajectory following concussion, with overall similar findings to the limited female collegiate student-athlete literature. Overall varying confidence rank classification was observed acutely. Our findings provide clinically-relevant insights for athletes, clinicians, researchers, and policymakers to inform efforts specific to females experiencing concussion.

Initial Mild Traumatic Brain Injury Characteristics and Recovery Patterns Among Females Across the United States Military Service Academies: A Report from the NCAA-DoD CARE Consortium

Mild traumatic brain injury (mTBI) has been described in the United States (US) military service academy cadet population, but female-specific characteristics and recovery outcomes are poorly characterized despite sex being a confounder. Our objective was to describe female cadets' initial characteristics, assessment performance, and return-to-activity outcomes post-mTBI. Female cadets (n = 472) from the four US military service academies who experienced a mTBI completed standardized mTBI assessments from pre-injury to acute initial injury and unrestricted return-to-duty (uRTD). Initial injury presentation characteristics (e.g., delayed symptoms, retrograde amnesia) and return-to-activity outcomes [i.e., return-to-learn, initiate return-to-duty protocol (iRTD), uRTD] were documented. Descriptive statistics summarized female cadets' injury characteristics, return-to-activity outcomes, and post-mTBI assessment performance change categorization (worsened, unchanged, improved) relative to pre-injury baseline using established change score confidence rank criteria for each assessment score. The median (interquartile range) days to return-to-learn (n = 157) was 7.0 (3.0-14.0), to iRTD (n = 412) was 14.7 (8.6-25.8), and to uRTD (n = 431) was 26.0 (17.7-41.8). The majority experienced worse SCAT total symptom severity (77.8%) and ImPACT reaction time (97.0%) acutely < 24-h versus baseline, but unchanged BESS total errors (75.2%), SAC total score (72%), BSI-18 total score (69.6%), and ImPACT verbal memory (62.3%), visual memory (58.4%), and visual motor speed (52.5%). We observed similar return-to-activity times in the present female cadet cohort relative to the existing female-specific literature. Confidence ranks categorizing post-mTBI performance were heterogenous and indicate multimodal assessments are necessary. Our findings provide clinically relevant insights to female cadets experiencing mTBI across the US service academies for stakeholders providing healthcare.

Baseline Normative and Test-Retest Reliability Data for Sideline Concussion Assessment Measures in Youth

Tools used for the identification, evaluation, and monitoring of concussion have not been sufficiently studied in youth or real-world settings. Normative and reliability data on sideline concussion assessment measures in the youth athlete population is needed. Pre-season normative data for 515 athletes (93.5% male) aged 5 to 16 on the Standardized Assessment of Concussion (SAC/SAC-Child), modified Balance Errors Scoring System (mBESS), Timed Tandem Gait (TTG), and the King-Devick Test (KDT) are provided. A total of 212 non-injured athletes repeated the measures post-season to assess test-retest reliability. Mean performance on the SAC-C, mBESS, TTG, and KDT tended to improve with age. KDT was the only measure that demonstrated good to excellent stability across age ranges (ICC = 0.758 to 0.941). Concentration was the only SAC/SAC-C subtest to demonstrate moderate test-retest stability (ICC = 0.503 to 0.706). TTG demonstrated moderate to good (ICC = 0.666 to 0.811) reliability. mBESS demonstrated poor to moderate reliability (ICC = -0.309 to 0.651). Commonly used measures of concussion vary regarding test-retest reliability in youth. The data support the use of at least annual sport concussion baseline assessments in the pediatric population to account for the evolution in performance as the child ages. Understanding the variation in the stability and the evolution of baseline performance will enable improved identification of possible injury.

Minimal Detectable Change for the ImPACT Subtests at Baseline

OBJECTIVE

To establish the minimal detectable change (MDC) of the subtests that comprise the composite scores from remotely administered Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) baselines.

METHOD

Remote ImPACT baseline data from 172 (male = 45, female = 127) National Collegiate Athletic Association Division I student-athletes from the 2020 and 2021 athletic preseasons were used to calculate the MDC at the 95%, 90%, and 80% confidence intervals (CIs) for all subtest scores used to generate the four core composite scores and the impulse control composite.

RESULTS

The MDCs for the verbal memory subtests at the 95% CI were 10.31 for word memory percent correct, 4.68 for symbol match total correct hidden, and 18.25 for three letters percentage correct. Visual memory subtest MDCs were 19.03 for design memory total percent correct and 4.90 for XO total correct memory. Visual motor speed subtest MDCs were 18.89 for XO total correct interference and 5.40 for three letters average counted correctly. Reaction time (RT) MDCs were 0.12 for XO average correct, 0.95 for symbol match average correct RT, and 0.28 for color match average correct. Impulse control MDCs were 5.97 for XO total incorrect and 1.15 for color match total commissions. One-way repeated measures MANOVA, repeated measures ANOVAs, and Wilcoxon signed-ranks test all suggested no significant difference between any subtests across two remote ImPACT baselines.

CONCLUSIONS

The ImPACT subtest scores did not significantly change between athletic seasons. Our study suggests the subtests be evaluated in conjunction with the composite scores to provide additional metrics for clinical interpretation.

Navigating uncertainty: exploring parents' knowledge of concussion management and neuropsychological baseline testing

Introduction

Parents play an important role in preventing and managing sport-related concussions among youth sport participants. Research indicates that parents understand the severity and consequences associated with the injury but gaps exist in their knowledge of its management. Neuropsychological baseline testing (NBT) is a modality that has gained interest in youth sport to purportedly better manage concussion injuries. Little is known about parents' perspectives on the use of NBT in the management process.

Methods

The present qualitative study used Protection Motivation Theory as a guiding framework and employed focus groups (N = 2) with parents (N = 11) to gain insight into parents' perceptions and experiences with concussion management, specifically focusing on NBT.

Results

Inductive Content Analysis developed a core theme of navigating uncertainty. Participants expressed uncertainty about the nature of concussion and its management process, where concussion was not always easy to identify, youth were not always reliable reporters, and there was no prescribed or proscribed path for recovery. Personal experience and concussion management policy provided participants with a degree of certainty in managing concussions. Participants gave NBT mixed reviews in potentially promoting greater certainty but also held reservations about its usefulness in concussion management.

Discussion

We discuss findings relative to existing knowledge and theory in youth sport concussion and identify implications for practice.

Psychometric Properties of Computerized Cognitive Tools and Standard Neuropsychological Tests Used to Assess Sport Concussion: A Systematic Review

Athletic programs are more frequently turning to computerized cognitive tools in order to increase efficiencies in concussion assessment. However, assessment using a traditional neuropsychological test battery may provide a more comprehensive and individualized evaluation. Our goal was to inform sport clinicians of the best practices for concussion assessment through a systematic literature review describing the psychometric properties of standard neuropsychological tests and computerized tools. We conducted our search in relevant databases including Ovid Medline, Web of Science, PsycINFO, and Scopus. Journal articles were included if they evaluated psychometric properties (e.g., reliability, sensitivity) of a cognitive assessment within pure athlete samples (up to 30 days post-injury). Searches yielded 4,758 unique results. Ultimately, 103 articles met inclusion criteria, all of which focused on adolescent or young adult participants. Test-retest reliability estimates ranged from .14 to .93 for computerized tools and .02 to .95 for standard neuropsychological tests, with strongest correlations on processing speed tasks for both modalities, although processing speed tasks were most susceptible to practice effects. Reliability was improved with a 2-factor model (processing speed and memory) and by aggregating multiple baseline exams, yet remained below acceptable limits for some studies. Sensitivity to decreased cognitive performance within 72 h of injury ranged from 45%-93% for computerized tools and 18%-80% for standard neuropsychological test batteries. The method for classifying cognitive decline (normative comparison, reliable change indices, regression-based methods) affected sensitivity estimates. Combining computerized tools and standard neuropsychological tests with the strongest psychometric performance provides the greatest value in clinical assessment. To this end, future studies should evaluate the efficacy of hybrid test batteries comprised of top-performing measures from both modalities.

Minimal Detectable Change for the ImPACT Test Administered Remotely

CONTEXT

Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is one of the most widely used computerized neurocognitive assessment batteries in athletics and serves as both a baseline and postinjury assessment. It has become increasingly popular to administer the ImPACT baseline test in an unsupervised remote environment; however, whether the lack of supervision affects the test-retest reliability is unknown.

OBJECTIVE

To establish the minimal detectable change (MDC) of composite scores from the ImPACT test when administered to National Collegiate Athletic Association Division I student-athletes in an unsupervised remote environment before 2 consecutive athletic seasons.

DESIGN

Cross-sectional study.

SETTING

Participants were provided with a unique link and detailed written instructions on how to complete the ImPACT test at home.

PATIENTS OR OTHER PARTICIPANTS

Division I student-athletes.

MAIN OUTCOME MEASURE(S)

Remote baseline ImPACT results from the 2020-2021 and 2021-2022 athletic seasons were analyzed. The MDC was calculated at the 95%, 90%, and 80% CIs for each of the ImPACT composite scores as well as the average and SD.

RESULTS

The MDC at the 95% CI was 18.6 for the verbal memory composite score, 24.44 for visual memory, 8.76 for visual motor, 0.14 for reaction time, and 6.13 for impulse control. One-way repeated-measures multivariate analysis of variance, repeated-measures analysis of variance, and Wilcoxon signed ranks tests suggested no difference in the composite scores and impulse control between time points.

CONCLUSIONS

The ImPACT composite scores and impulse control did not change between the 2 remote testing time points when administered approximately 1 year apart. Our study suggests that the MDC serves as a clinician's guide for evaluating changes in ImPACT baseline scores and in making clinical judgments on sport-related concussion when the test is administered at home.

Baseline concussion assessment performance by sex in military service academy rugby players: findings from the CARE Consortium

BACKGROUND

Normative student-athlete concussion assessment data may not be appropriate for service academy members (SAMs), particularly rugby players, because of the uniqueness of their academic/military training environment. Having accurate baseline data for this population is important because of their high risk for concussion and frequent lack of assigned sports medicine professional. The primary purpose of this study was to characterise baseline performance on a concussion assessment battery, with secondary purpose to determine effect of sex and concussion history on these measures among SAM rugby players.

METHODS

601 rugby-playing SAMs (19.3±1.5 years, 37.9% female) completed baseline concussion assessments: the Sport Concussion Assessment Tool (SCAT) Symptom and Symptom Severity Checklist, Standard Assessment of Concussion (SAC) and a neuropsychological test (either ImPACT (Immediate Post Concussion Assessment and Cognitive Testing) or ANAM (Automated Neuropsychological Assessment Metrics)). Groups were compared using an independent samples t-test or Mann-Whitney U test. A 2 (sex) × 2 (concussion history) ANOVA was conducted to determine the effects of sex and concussion history on outcomes.

RESULTS

Women reported greater SCAT total symptoms (3.3 vs 2.8, p<0.001, r=0.143) and symptom severities (5.7 vs 4.3, p<0.001, r=0.139), and performed worse on ImPACT Visual Memory (79.3 vs 82.6, p=0.002, r=0.144) than men. Women performed better than men on SAC (28.0 vs 27.7, p=0.03, r=0.088), ImPACT Reaction Time Composite (0.59 vs 0.61, p=0.04, r=0.092) and ANAM Code Substitution Delayed (64.3 vs 61.5, p=0.04, d=0.433). Individuals with a history of concussion reported lower ImPACT Symptom Severity (2.6 vs 4.2, p=0.02, r=0.110). There was no interaction between concussion history and sex on outcomes.

CONCLUSIONS

These findings provide reference data for SAM rugby players on baseline assessments and to help in clinical decision-making when managing sports-related concussion in absence of baseline data.

Proteomic Discovery of Noninvasive Biomarkers Associated With Sport-Related Concussions

BACKGROUND AND OBJECTIVES

Sport-related concussions affect millions of individuals across the United States each year, and current techniques to diagnose and monitor them rely largely on subjective measures. Our goal was to discover and validate objective, quantifiable noninvasive biomarkers with the potential to be used in sport-related concussion diagnosis.

METHODS

Urine samples from a convenience series of healthy control collegiate athletes who had not sustained a concussion and athletes who sustained a concussion as diagnosed by a sports medicine physician within 7 days were collected prospectively and studied. Participants also completed an instrumented single-task gait analysis as a functional measure. Participants were recruited from a single collegiate athletic program and were ≥18 years of age and were excluded if they had a concomitant injury, active psychiatric conditions, or preexisting neurologic disorders. Using Tandem Mass Tags (TMT) mass spectroscopy and ELISA, we identified and validated urinary biomarkers of concussion.

RESULTS

Forty-eight control and 47 age- and sex-matched athletes with concussion were included in the study (51.6% female, 48.4% male, average age 19.6 years). Participants represented both contact and noncontact sports. All but 1 of the postconcussion participants reported experiencing symptoms at the time of data collection. Insulin-like growth factor 1 (IGF-1) and IGF binding protein 5 (IGFBP5) were downregulated in the urine of athletes with concussions compared to healthy controls. Multivariable risk algorithms developed to predict the probability of sport-related concussion showed that IGF-1 multiplexed with single-task gait velocity predicts concussion risk across a range of postinjury time points (area under the curve [AUC] 0.786, 95% confidence interval [CI] 0.690-0.884). When IGF-1 and IGFBP5 are multiplexed with single-task gait velocity, they accurately distinguish between healthy controls and individuals with concussion at acute time points (AUC 0.835, 95% CI 0.701-0.968, p < 0.001).

DISCUSSION

These noninvasive biomarkers, discovered in an objective and validated manner, may be useful in diagnosing and monitoring sport-related concussions in both acute phases of injury and several days after injury.

TRIAL REGISTRATION INFORMATION

ClinicalTrials.gov Identifier: NCT02354469 (submitted February 2015, first patient enrolled August 2015).

CLASSIFICATION OF EVIDENCE

This study provides Class III evidence that urinary IGF-1 and IGFBP5 multiplexed with single-task gait velocity may be useful in diagnosing sport-related concussion.

Measuring test-retest reliability (TRR) of AMSTAR provides moderate to perfect agreement - a contribution to the discussion of the importance of TRR in relation to the psychometric properties of assessment tools

Background

Systematic Reviews (SRs) can build the groundwork for evidence-based health care decision-making. A sound methodological quality of SRs is crucial. AMSTAR (A Measurement Tool to Assess Systematic Reviews) is a widely used tool developed to assess the methodological quality of SRs of randomized controlled trials (RCTs). Research shows that AMSTAR seems to be valid and reliable in terms of interrater reliability (IRR), but the test retest reliability (TRR) of AMSTAR has never been investigated. In our study we investigated the TRR of AMSTAR to evaluate the importance of its measurement and contribute to the discussion of the measurement properties of AMSTAR and other quality assessment tools.

Methods

Seven raters at three institutions independently assessed the methodological quality of SRs in the field of occupational health with AMSTAR. Between the first and second ratings was a timespan of approximately two years. Answers were dichotomized, and we calculated the TRR of all raters and AMSTAR items using Gwet’s AC1 coefficient. To investigate the impact of variation in the ratings over time, we obtained summary scores for each review.

Results

AMSTAR item 4 (Was the status of publication used as an inclusion criterion?) provided the lowest median TRR of 0.53 (moderate agreement). Perfect agreement of all reviewers was detected for AMSTAR-item 1 with a Gwet’s AC1 of 1, which represented perfect agreement. The median TRR of the single raters varied between 0.69 (substantial agreement) and 0.89 (almost perfect agreement). Variation of two or more points in yes-scored AMSTAR items was observed in 65% (73/112) of all assessments.

Conclusions

The high variation between the first and second AMSTAR ratings suggests that consideration of the TRR is important when evaluating the psychometric properties of AMSTAR.. However, more evidence is needed to investigate this neglected issue of measurement properties. Our results may initiate discussion of the importance of considering the TRR of assessment tools. A further examination of the TRR of AMSTAR, as well as other recently established rating tools such as AMSTAR 2 and ROBIS (Risk Of Bias In Systematic reviews), would be useful.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01231-y.