Optimizing Components of the Sport Concussion Assessment Tool for Acute Concussion Assessment

Optimizing baseline and post-concussion assessments through identification, confirmation, and equivalence of latent factor structures: Findings from the NCAA-DoD CARE Consortium

Objective: Concussion evaluations use a multidimensional assessment to evaluate unique patient function dimensions (e.g., subjective symptoms differ from balance assessments), but the overarching latent factor structure has not been empirically substantiated. Our objective was to determine the cumulative latent factor structure of pre-injury baseline and acute (<48-h) post-concussion assessment battery outcomes, and determine measurement equivalence among common factors in collegiate student-athletes. Methods: Collegiate student-athletes at baseline (n = 21,865) and post-concussion (n = 1,537) across 25-institutions completed standardized assessments. Individual items were used from the baseline and post-concussion assessments and consisted of: Sport Concussion Assessment Tool, Brief Symptom Inventory-18, Standardized Assessment of Concussion, Balance Error Scoring System, Immediate Post-Concussion Assessment and Cognitive Test, and vestibular-ocular motor screening. Exploratory factor analysis was used on half the baseline data, and confirmatory factor analysis on the remaining baseline data and post-concussion data separately. Measurement equivalence was assessed between sex, sport contact classification, concussion history, and time. Results: A 10-factor exploratory model was established and comprised of: depression, somatic, vestibulo-ocular, headache, postural stability, neurocognition, emotional, fatigue, cognitive, consciousness clouding. The 10-factor model was confirmed at baseline and post-concussion with strong measurement equivalence between timepoints. Strong to strict measurement equivalence was observed for sex, sport contact classification, and concussion history at both timepoints separately. Conclusion: Our findings established a robust 10-factor latent factor model equivalent across timepoints and common factors among healthy and concussed collegiate athletes. Clinicians can use these findings to target specific factors while reducing redundant elements to provide efficient, comprehensive post-concussion assessments.

National Collegiate Athletic Association athletic trainers' response to the Arrington settlement: management, compliance, and practice patterns

The primary purpose of this study was to assess Athletic Trainers' (ATs) report of NCAA member institution compliance with the Arrington settlement, the concussion lawsuit vs. the NCAA, and to elucidate compliance predictors. A secondary purpose was to provide a contemporary concussion management clinical practice pattern description among NCAA collegiate athletic trainers. Head Athletic Trainers from NCAA Division I, II, and III completed an electronic questionnaire in August 2020 regarding their institution's response to the Arrington Settlement and their current concussion management clinical practice patterns. The 37-item questionnaire included AT and institution demographics, current concussion management policies, and response to the Arrington settlement with a specific focus on the five settlement requirements. An overall compliance score on the five requirements, compliance on the individual requirements, and concussion management practices are reported with descriptives. Regression was used to identify specific predictors of both overall and individual settlement requirements. An ANOVA compared compliance by NCAA division level. Being pressured to be non-compliant was assessed between sexes by a chi-square. There were 223 respondents (21.8%), and overall compliance was high (4.1 ± 0.7) with the five required Arrington Settlement components. Settlement requirement 1, pre-season baseline testing, and requirement 5, presence of trained personnel at all contact sport practices, had the lowest compliance rates at 44.8% and 73.3%, respectively. The number of sports the institution offered was the only significant predictor of each requirement. There was no difference in compliance between NCAA divisions. Although the overall rate of being non-compliant pressure was low (13.8%), females were 3.28x more likely report being pressured than males. NCAA institutions are generally compliant with the Arrington settlement; however, lack of clarity in the requirements, particularly requirement 1, raises potential concerns. Concussion management practices continue to incorporate multifaceted approaches and are largely consistent with current best practices.

Test-Retest Reliability and Efficacy of Individual Symptoms in Concussion Management

OBJECTIVE

(1) To determine test-retest reliability of individual Sport Concussion Assessment Tool-Third Edition (SCAT-3) symptom scores and symptom severity scores, (2) to examine the specificity/sensitivity of individual SCAT-3 symptom severity scores acutely (24-48 hours) postconcussion, and (3) to develop a model of symptoms best able to differentiate concussed from nonconcussed student athletes and cadets.

DESIGN

Prospective, longitudinal, and cross-sectional.

SETTING

Twenty-six civilian schools and 3 US service academies.

PARTICIPANTS

Collegiate student athletes (n = 5519) and cadets (n = 5359) from the National Collegiate Athletic Association-Department of Defense Grand Alliance: Concussion Assessment, Research and Education Consortium, including 290 student athletes and 205 cadets, assessed 24 to 48 hours postconcussion.

INDEPENDENT VARIABLES

Concussed and nonconcussed student athlete and cadet groups.

MAIN OUTCOME MEASURES

Sport Concussion Assessment Tool-Third Edition individual symptom severity scores, total symptom scores, and symptom severity scores.

RESULTS

Results indicated poor test-retest reliability across all symptom scores (intraclass correlation coefficient = 0.029-0.331), but several individual symptoms had excellent predictive capability in discriminating concussed from nonconcussed participants (eg, headache, pressure in the head, and don't feel right had area under the curve >0.8, sensitivity >70%, and specificity >85%) regardless of baseline testing. These symptoms were consistent with Chi-square Automatic Interaction Detector classification trees with the highest mean probability.

CONCLUSIONS

Findings support the excellent diagnostic accuracy of honest symptom reporting, notwithstanding the known limitations in symptom underreporting, and suggest that there may be added value in examining individual symptoms rather than total symptom scores and symptom severity scores alone. Finally, findings suggest that baseline testing is not necessary for interpreting postconcussion symptom scores.

MICK (Mobile Integrated Cognitive Kit) app: Feasibility of an accessible tablet-based rapid picture and number naming task for concussion assessment in a division 1 college football cohort

Although visual symptoms are common following concussion, quantitative measures of visual function are missing from concussion evaluation protocols on the athletic sideline. For the past half century, rapid automatized naming (RAN) tasks have demonstrated promise as quantitative neuro-visual assessment tools in the setting of head trauma and other disorders but have been previously limited in accessibility and scalability. The Mobile Interactive Cognitive Kit (MICK) App is a digital RAN test that can be downloaded on most mobile devices and can therefore provide a quantitative measure of visual function anywhere, including the athletic sideline. This investigation examined the feasibility of MICK App administration in a cohort of Division 1 college football players. Participants (n = 82) from a National Collegiate Athletic Association (NCAA) Division 1 football team underwent baseline testing on the MICK app. Total completion times of RAN tests on the MICK app were recorded; magnitudes of best time scores and between-trial learning effects were determined by paired t-test. Consistent with most timed performance measures, there were significant learning effects between the two baseline trials for both RAN tasks on the MICK app: Mobile Universal Lexicon Evaluation System (MULES) (p < 0.001, paired t-test, mean improvement 13.3 s) and the Staggered Uneven Number (SUN) (p < 0.001, mean improvement 3.3 s). This study demonstrated that the MICK App can be feasibly administered in the setting of pre-season baseline testing in a Division I environment. These data provide a foundation for post-injury sideline testing that will include comparison to baseline in the setting of concussion.

Traumatic brain injury: progress and challenges in prevention, clinical care, and research

Traumatic brain injury (TBI) has the highest incidence of all common neurological disorders, and poses a substantial public health burden. TBI is increasingly documented not only as an acute condition but also as a chronic disease with long-term consequences, including an increased risk of late-onset neurodegeneration. The first Lancet Neurology Commission on TBI, published in 2017, called for a concerted effort to tackle the global health problem posed by TBI. Since then, funding agencies have supported research both in high-income countries (HICs) and in low-income and middle-income countries (LMICs). In November 2020, the World Health Assembly, the decision-making body of WHO, passed resolution WHA73.10 for global actions on epilepsy and other neurological disorders, and WHO launched the Decade for Action on Road Safety plan in 2021. New knowledge has been generated by large observational studies, including those conducted under the umbrella of the International Traumatic Brain Injury Research (InTBIR) initiative, established as a collaboration of funding agencies in 2011. InTBIR has also provided a huge stimulus to collaborative research in TBI and has facilitated participation of global partners. The return on investment has been high, but many needs of patients with TBI remain unaddressed. This update to the 2017 Commission presents advances and discusses persisting and new challenges in prevention, clinical care, and research.

In LMICs, the occurrence of TBI is driven by road traffic incidents, often involving vulnerable road users such as motorcyclists and pedestrians. In HICs, most TBI is caused by falls, particularly in older people (aged ≥65 years), who often have comorbidities. Risk factors such as frailty and alcohol misuse provide opportunities for targeted prevention actions. Little evidence exists to inform treatment of older patients, who have been commonly excluded from past clinical trials—consequently, appropriate evidence is urgently required. Although increasing age is associated with worse outcomes from TBI, age should not dictate limitations in therapy. However, patients injured by low-energy falls (who are mostly older people) are about 50% less likely to receive critical care or emergency interventions, compared with those injured by high-energy mechanisms, such as road traffic incidents.

Mild TBI, defined as a Glasgow Coma sum score of 13–15, comprises most of the TBI cases (over 90%) presenting to hospital. Around 50% of adult patients with mild TBI presenting to hospital do not recover to pre-TBI levels of health by 6 months after their injury. Fewer than 10% of patients discharged after presenting to an emergency department for TBI in Europe currently receive follow-up. Structured follow-up after mild TBI should be considered good practice, and urgent research is needed to identify which patients with mild TBI are at risk for incomplete recovery.

The selection of patients for CT is an important triage decision in mild TBI since it allows early identification of lesions that can trigger hospital admission or life-saving surgery. Current decision making for deciding on CT is inefficient, with 90–95% of scanned patients showing no intracranial injury but being subjected to radiation risks. InTBIR studies have shown that measurement of blood-based biomarkers adds value to previously proposed clinical decision rules, holding the potential to improve efficiency while reducing radiation exposure. Increased concentrations of biomarkers in the blood of patients with a normal presentation CT scan suggest structural brain damage, which is seen on MR scanning in up to 30% of patients with mild TBI. Advanced MRI, including diffusion tensor imaging and volumetric analyses, can identify additional injuries not detectable by visual inspection of standard clinical MR images. Thus, the absence of CT abnormalities does not exclude structural damage—an observation relevant to litigation procedures, to management of mild TBI, and when CT scans are insufficient to explain the severity of the clinical condition.

Although blood-based protein biomarkers have been shown to have important roles in the evaluation of TBI, most available assays are for research use only. To date, there is only one vendor of such assays with regulatory clearance in Europe and the USA with an indication to rule out the need for CT imaging for patients with suspected TBI. Regulatory clearance is provided for a combination of biomarkers, although evidence is accumulating that a single biomarker can perform as well as a combination. Additional biomarkers and more clinical-use platforms are on the horizon, but cross-platform harmonisation of results is needed. Health-care efficiency would benefit from diversity in providers.

In the intensive care setting, automated analysis of blood pressure and intracranial pressure with calculation of derived parameters can help individualise management of TBI. Interest in the identification of subgroups of patients who might benefit more from some specific therapeutic approaches than others represents a welcome shift towards precision medicine. Comparative-effectiveness research to identify best practice has delivered on expectations for providing evidence in support of best practices, both in adult and paediatric patients with TBI.

Progress has also been made in improving outcome assessment after TBI. Key instruments have been translated into up to 20 languages and linguistically validated, and are now internationally available for clinical and research use. TBI affects multiple domains of functioning, and outcomes are affected by personal characteristics and life-course events, consistent with a multifactorial bio-psycho-socio-ecological model of TBI, as presented in the US National Academies of Sciences, Engineering, and Medicine (NASEM) 2022 report. Multidimensional assessment is desirable and might be best based on measurement of global functional impairment. More work is required to develop and implement recommendations for multidimensional assessment. Prediction of outcome is relevant to patients and their families, and can facilitate the benchmarking of quality of care. InTBIR studies have identified new building blocks (eg, blood biomarkers and quantitative CT analysis) to refine existing prognostic models. Further improvement in prognostication could come from MRI, genetics, and the integration of dynamic changes in patient status after presentation.

Neurotrauma researchers traditionally seek translation of their research findings through publications, clinical guidelines, and industry collaborations. However, to effectively impact clinical care and outcome, interactions are also needed with research funders, regulators, and policy makers, and partnership with patient organisations. Such interactions are increasingly taking place, with exemplars including interactions with the All Party Parliamentary Group on Acquired Brain Injury in the UK, the production of the NASEM report in the USA, and interactions with the US Food and Drug Administration. More interactions should be encouraged, and future discussions with regulators should include debates around consent from patients with acute mental incapacity and data sharing. Data sharing is strongly advocated by funding agencies. From January 2023, the US National Institutes of Health will require upload of research data into public repositories, but the EU requires data controllers to safeguard data security and privacy regulation. The tension between open data-sharing and adherence to privacy regulation could be resolved by cross-dataset analyses on federated platforms, with the data remaining at their original safe location. Tools already exist for conventional statistical analyses on federated platforms, however federated machine learning requires further development. Support for further development of federated platforms, and neuroinformatics more generally, should be a priority.

This update to the 2017 Commission presents new insights and challenges across a range of topics around TBI: epidemiology and prevention (section 1 ); system of care (section 2 ); clinical management (section 3 ); characterisation of TBI (section 4 ); outcome assessment (section 5 ); prognosis (Section 6 ); and new directions for acquiring and implementing evidence (section 7 ). Table 1 summarises key messages from this Commission and proposes recommendations for the way forward to advance research and clinical management of TBI.

Phybrata Sensors and Machine Learning for Enhanced Neurophysiological Diagnosis and Treatment

Concussion injuries remain a significant public health challenge. A significant unmet clinical need remains for tools that allow related physiological impairments and longer-term health risks to be identified earlier, better quantified, and more easily monitored over time. We address this challenge by combining a head-mounted wearable inertial motion unit (IMU)-based physiological vibration acceleration (“phybrata”) sensor and several candidate machine learning (ML) models. The performance of this solution is assessed for both binary classification of concussion patients and multiclass predictions of specific concussion-related neurophysiological impairments. Results are compared with previously reported approaches to ML-based concussion diagnostics. Using phybrata data from a previously reported concussion study population, four different machine learning models (Support Vector Machine, Random Forest Classifier, Extreme Gradient Boost, and Convolutional Neural Network) are first investigated for binary classification of the test population as healthy vs. concussion (Use Case 1). Results are compared for two different data preprocessing pipelines, Time-Series Averaging (TSA) and Non-Time-Series Feature Extraction (NTS). Next, the three best-performing NTS models are compared in terms of their multiclass prediction performance for specific concussion-related impairments: vestibular, neurological, both (Use Case 2). For Use Case 1, the NTS model approach outperformed the TSA approach, with the two best algorithms achieving an F1 score of 0.94. For Use Case 2, the NTS Random Forest model achieved the best performance in the testing set, with an F1 score of 0.90, and identified a wider range of relevant phybrata signal features that contributed to impairment classification compared with manual feature inspection and statistical data analysis. The overall classification performance achieved in the present work exceeds previously reported approaches to ML-based concussion diagnostics using other data sources and ML models. This study also demonstrates the first combination of a wearable IMU-based sensor and ML model that enables both binary classification of concussion patients and multiclass predictions of specific concussion-related neurophysiological impairments.

Diagnostic accuracy and reliability of sideline concussion evaluation: a prospective, case-controlled study in college athletes comparing newer tools and established tests

OBJECTIVE

To assess diagnostic accuracy and reliability of sideline concussion tests in college athletes.

METHODS

Athletes completed baseline concussion tests including Post-Concussion Symptom Scale, Standardised Assessment of Concussion (SAC), modified Balance Error Scoring System (m-BESS), King-Devick test and EYE-SYNC Smooth Pursuits. Testing was repeated in athletes diagnosed acutely with concussion and compared to a matched teammate without concussion.

RESULTS

Data were collected on 41 concussed athletes and 41 matched controls. Test-retest reliability for symptom score and symptom severity assessed using control athletes was 0.09 (-0.70 to 0.88) and 0.08 (-1.00 to 1.00) (unweighted kappa). Intraclass correlations were SAC 0.33 (-0.02 to 0.61), m-BESS 0.33 (-0.2 to 0.60), EYE-SYNC Smooth Pursuit tangential variability 0.70 (0.50 to 0.83), radial variability 0.47 (0.19 to 0.69) and King-Devick test 0.71 (0.49 to 0.84). The maximum identified sensitivity/specificity of each test for predicting clinical concussion diagnosis was: symptom score 81%/94% (3-point increase), symptom severity score 91%/81% (3-point increase), SAC 44%/72% (2-point decline), m-BESS 40%/92% (5-point increase), King-Devick 85%/76% (any increase in time) and EYE-SYNC Smooth Pursuit tangential variability 48%/58% and radial variability 52%/61% (any increase). Adjusted area under the curve was: symptom score 0.95 (0.89, 0.99), symptom severity 0.95 (95% CI 0.88 to 0.99), SAC 0.66 (95% CI 0.54 to 0.79), m-BESS 0.71 (0.60, 0.83), King-Devick 0.78 (0.69, 0.87), radial variability 0.47 (0.34, 0.59), tangential variability 0.41 (0.30, 0.54) CONCLUSION: Test-retest reliability of most sideline concussion tests was poor in uninjured athletes, raising concern about the accuracy of these tests to detect new concussion. Symptom score/severity had the greatest sensitivity and specificity, and of the objective tests, the King-Devick test performed best.

Developing Insights for Possible and Probable Acute Concussions Using Cluster Analysis

Few studies have analyzed the Sport Concussion Assessment Tool's (SCAT) utility among athletes whose concussion assessment is challenging. Using a previously published algorithm, we identified possible and probable concussions at <6 h (n = 393 males, n = 265 females) and 24-48 h (n = 323 males, n = 236 females) post-injury within collegiate student-athletes and cadets from the Concussion Assessment, Research, and Education (CARE) Consortium. We applied cluster analysis to characterize performance on the Standard Assessment of Concussion (SAC), Balance Error Scoring System (BESS), and the SCAT symptom checklist for these athletes. Among the cluster sets that best separated acute concussions and normal performances, total symptom number raw score and change and post-traumatic migraine raw score and change score were the most frequent clustering variables across males and females at <6 h and 24-48 h. Similarly, total symptom number raw score and change score and post-traumatic migraine raw score and change score were most significantly different between clusters for males and females at <6 h and 24-48 h. Our results suggest that clinicians should focus on total symptom number, post-traumatic migraine symptoms, and cognitive-fatigue symptoms when assessing possible and probable concussions, followed by the SAC and BESS scores.

Detailed description of Division I ice hockey concussions: Findings from the NCAA and Department of Defense CARE Consortium

OBJECTIVE

Since concussion is the most common injury in ice hockey, the objective of the current study was to elucidate risk factors, specific mechanisms, and clinical presentations of concussion in men's and women's ice hockey.

METHODS

Ice hockey players from 5 institutions participating in the Concussion Assessment, Research, and Education Consortium were eligible for the current study. Participants who sustained a concussion outside of this sport were excluded. There were 332 (250 males, 82 females) athletes who participated in ice hockey, and 47 (36 males, 11 females) who sustained a concussion.

RESULTS

Previous concussion (odds ratio (OR) = 2.00; 95% confidence interval (95% CI): 1.02‒3.91) was associated with increased incident concussion odds, while wearing a mouthguard was protective against incident concussion (OR = 0.43; 95%CI: 0.22‒0.85). Overall, concussion mechanisms did not significantly differ between sexes. There were specific differences in how concussions presented clinically across male and female ice hockey players, however. Females (9.09%) were less likely than males (41.67%) to have a delayed symptom onset (p = 0.045). Additionally, females took significantly longer to reach asymptomatic (p = 0.015) and return-to-play clearance (p = 0.005). Within the first 2 weeks post-concussion, 86.11% of males reached asymptomatic, while only 45.50% of females reached the same phase of recovery. Most males (91.67%) were cleared for return to play within 3 weeks of their concussion, compared to less than half (45.50%) of females.

CONCLUSION

The current study proposes possible risk factors, mechanisms, and clinical profiles to be validated in future concussions studies with larger female sample sizes. Understanding specific risk factors, concussion mechanisms, and clinical profiles of concussion in collegiate ice hockey may generate ideas for future concussion prevention or intervention studies.

Sensitivity and Specificity of Computer-Based Neurocognitive Tests in Sport-Related Concussion: Findings from the NCAA-DoD CARE Consortium

BACKGROUND

To optimally care for concussed individuals, a multi-dimensional approach is critical and a key component of this assessment in the athletic environment is computer-based neurocognitive testing. However, there continues to be concerns about the reliability and validity of these testing tools. The purpose of this study was to determine the sensitivity and specificity of three common computer-based neurocognitive tests (Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT], CNS Vital Signs, and CogState Computerized Assessment Tool [CCAT]), to provide guidance on their clinical utility.

METHODS

This study analyzed assessments from a cohort of collegiate athletes and non-varsity cadets from the NCAA-DoD CARE Consortium. The data were collected from 2014-2018. Study participants were divided into two testing groups [concussed, n = 1414 (baseline/24-48 h) and healthy, n = 8305 (baseline/baseline)]. For each test type, change scores were calculated for the components of interest. Then, the Normative Change method, which used normative data published in a similar cohort, and the Reliable Change Index (RCI) method were used to determine if the change scores were significant.

RESULTS

Using the Normative Change method, ImPACT performed best with an 87.5%-confidence interval and 1 number of components failed (NCF; sensitivity = 0.583, specificity = 0.625, F1 = 0.308). CNS Vital Signs performed best with a 90%-confidence interval and 1 NCF (sensitivity = 0.587, specificity = 0.532, F1 = 0.314). CCAT performed best when using a 75%-confidence interval and 2 NCF (sensitivity = 0.513, specificity = 0.715, F1 = 0.290). When using the RCI method, ImPACT performed best with an 87.5%-confidence interval and 1 NCF (sensitivity = 0.626, specificity = 0.559, F1 = 0.297).

CONCLUSION

When considering all three computer-based neurocognitive tests, the overall low sensitivity and specificity results provide additional evidence for the use of a multi-dimensional assessment for concussion diagnosis, including symptom evaluation, postural control assessment, neuropsychological status, and other functional assessments.

Physiological Vibration Acceleration (Phybrata) Sensor Assessment of Multi-System Physiological Impairments and Sensory Reweighting Following Concussion

Objective

To assess the utility of a head-mounted wearable inertial motion unit (IMU)-based physiological vibration acceleration (“phybrata”) sensor to support the clinical diagnosis of concussion, classify and quantify specific concussion-induced physiological system impairments and sensory reweighting, and track individual patient recovery trajectories.

Methods

Data were analyzed from 175 patients over a 12-month period at three clinical sites. Comprehensive clinical concussion assessments were first completed for all patients, followed by testing with the phybrata sensor. Phybrata time series data and spatial scatter plots, eyes open (Eo) and eyes closed (Ec) phybrata powers, average power (Eo+Ec)/2, Ec/Eo phybrata power ratio, time-resolved phybrata spectral density (TRPSD) distributions, and receiver operating characteristic (ROC) curves are compared for individuals with no objective impairments and those clinically diagnosed with concussions and accompanying vestibular impairment, other neurological impairment, or both vestibular and neurological impairments. Finally, pre- and post-injury phybrata case report results are presented for a participant who was diagnosed with a concussion and subsequently monitored during treatment, rehabilitation, and return-to-activity clearance.

Results

Phybrata data demonstrate distinct features and patterns for individuals with no discernable clinical impairments, diagnosed vestibular pathology, and diagnosed neurological pathology. ROC curves indicate that the average power (Eo+Ec)/2 may be utilized to support clinical diagnosis of concussion, while Eo and Ec/Eo may be utilized as independent measures to confirm accompanying neurological and vestibular impairments, respectively. All 3 measures demonstrate area under the curve (AUC), sensitivity, and specificity above 90% for their respective diagnoses. Phybrata spectral analyses demonstrate utility for quantifying the severity of concussion-induced physiological impairments, sensory reweighting, and subsequent monitoring of improvements throughout treatment and rehabilitation.

Conclusion

Phybrata testing assists with objective concussion diagnosis and provides an important adjunct to standard concussion assessment tools by objectively ascertaining neurological and vestibular impairments, guiding targeted rehabilitation strategies, monitoring recovery, and assisting with return-to-sport/work/learn decision-making.