1
|
Wilmoth K, Brett BL, Emmert NA, Cook CM, Schaffert J, Caze T, Kotsonis T, Cusick M, Solomon G, Resch JE, Cullum CM, Nelson LD, McCrea M. Psychometric Properties of Computerized Cognitive Tools and Standard Neuropsychological Tests Used to Assess Sport Concussion: A Systematic Review. Neuropsychol Rev 2023; 33:675-692. [PMID: 36040610 DOI: 10.1007/s11065-022-09553-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 05/03/2022] [Indexed: 11/29/2022]
Abstract
Athletic programs are more frequently turning to computerized cognitive tools in order to increase efficiencies in concussion assessment. However, assessment using a traditional neuropsychological test battery may provide a more comprehensive and individualized evaluation. Our goal was to inform sport clinicians of the best practices for concussion assessment through a systematic literature review describing the psychometric properties of standard neuropsychological tests and computerized tools. We conducted our search in relevant databases including Ovid Medline, Web of Science, PsycINFO, and Scopus. Journal articles were included if they evaluated psychometric properties (e.g., reliability, sensitivity) of a cognitive assessment within pure athlete samples (up to 30 days post-injury). Searches yielded 4,758 unique results. Ultimately, 103 articles met inclusion criteria, all of which focused on adolescent or young adult participants. Test-retest reliability estimates ranged from .14 to .93 for computerized tools and .02 to .95 for standard neuropsychological tests, with strongest correlations on processing speed tasks for both modalities, although processing speed tasks were most susceptible to practice effects. Reliability was improved with a 2-factor model (processing speed and memory) and by aggregating multiple baseline exams, yet remained below acceptable limits for some studies. Sensitivity to decreased cognitive performance within 72 h of injury ranged from 45%-93% for computerized tools and 18%-80% for standard neuropsychological test batteries. The method for classifying cognitive decline (normative comparison, reliable change indices, regression-based methods) affected sensitivity estimates. Combining computerized tools and standard neuropsychological tests with the strongest psychometric performance provides the greatest value in clinical assessment. To this end, future studies should evaluate the efficacy of hybrid test batteries comprised of top-performing measures from both modalities.
Collapse
Affiliation(s)
- Kristin Wilmoth
- Departments of Psychiatry and Physical Medicine & Rehabilitation, UT Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX, 75390-9055, USA.
| | - Benjamin L Brett
- Departments of Neurosurgery and Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Natalie A Emmert
- Department of Neurology, University at Buffalo, State University of New York, Buffalo, NY, USA
| | - Carolyn M Cook
- Department of Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Jeffrey Schaffert
- Department of Psychiatry, UT Southwestern Medical Center, Dallas, TX, USA
| | - Todd Caze
- Department of Psychiatry, UT Southwestern Medical Center, Dallas, TX, USA
| | - Thomas Kotsonis
- Department of Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Margaret Cusick
- Department of Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Gary Solomon
- Player Health and Safety Department, National Football League and Department of Neurosurgery, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - Jacob E Resch
- Department of Kinesiology, University of Virginia, Charlottesville, VA, USA
| | - C Munro Cullum
- Departments of Psychiatry, Neurology, and Neurological Surgery, UT Southwestern Medical Center, Dallas, TX, USA
| | - Lindsay D Nelson
- Departments of Neurosurgery and Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Michael McCrea
- Departments of Neurosurgery and Neurology, Medical College of Wisconsin, Milwaukee, WI, USA
| |
Collapse
|
2
|
Schneider ALC, Huie JR, Boscardin WJ, Nelson L, Barber JK, Yaffe K, Diaz-Arrastia R, Ferguson AR, Kramer J, Jain S, Temkin N, Yuh E, Manley GT, Gardner RC. Cognitive Outcome 1 Year After Mild Traumatic Brain Injury: Results From the TRACK-TBI Study. Neurology 2022; 98:e1248-e1261. [PMID: 35173018 PMCID: PMC8967334 DOI: 10.1212/wnl.0000000000200041] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 01/03/2022] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND AND OBJECTIVES The objectives of this study were to develop and establish concurrent validity of a clinically relevant definition of poor cognitive outcome 1 year after mild traumatic brain injury (mTBI), to compare baseline characteristics across cognitive outcome groups, and to determine whether poor 1-year cognitive outcome can be predicted by routinely available baseline clinical variables. METHODS Prospective cohort study included 656 participants ≥17 years of age presenting to level 1 trauma centers within 24 hours of mTBI (Glasgow Coma Scale score 13-15) and 156 demographically similar healthy controls enrolled in the Transforming Research and Clinical Knowledge in TBI (TRACK-TBI) study. Poor 1-year cognitive outcome was defined as cognitive impairment (below the ninth percentile of normative data on ≥2 cognitive tests), cognitive decline (change score [1-year score minus best 2-week or 6-month score] exceeding the 90% reliable change index on ≥2 cognitive tests), or both. Associations of poor 1-year cognitive outcome with 1-year neurobehavioral outcomes were performed to establish concurrent validity. Baseline characteristics were compared across cognitive outcome groups, and backward elimination logistic regression was used to build a prediction model. RESULTS Mean age of participants with mTBI was 40.2 years; 36.6% were female; 76.6% were White. Poor 1-year cognitive outcome was associated with worse 1-year functional outcome, more neurobehavioral symptoms, greater psychological distress, and lower satisfaction with life (all p < 0.05), establishing concurrent validity. At 1 year, 13.5% of participants with mTBI had a poor cognitive outcome vs 4.5% of controls (p = 0.003). In univariable analyses, poor 1-year cognitive outcome was associated with non-White race, lower education, lower income, lack of health insurance, hyperglycemia, preinjury depression, and greater injury severity (all p < 0.05). The final multivariable prediction model included education, health insurance, preinjury depression, hyperglycemia, and Rotterdam CT score ≥3 and achieved an area under the curve of 0.69 (95% CI 0.62-0.75) for the prediction of a poor 1-year cognitive outcome, with each variable associated with >2-fold increased odds of poor 1-year cognitive outcome. DISCUSSION Poor 1-year cognitive outcome is common, affecting 13.5% of patients with mTBI vs 4.5% of controls. These results highlight the need for better understanding of mechanisms underlying poor cognitive outcome after mTBI to inform interventions to optimize cognitive recovery.
Collapse
Affiliation(s)
- Andrea L C Schneider
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - J Russell Huie
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - W John Boscardin
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Lindsay Nelson
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Jason K Barber
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Kristine Yaffe
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Ramon Diaz-Arrastia
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Adam R Ferguson
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Joel Kramer
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Sonia Jain
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Nancy Temkin
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Esther Yuh
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Geoffrey T Manley
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla
| | - Raquel C Gardner
- From the Department of Neurology (A.L.C.S., R.D.-A.), University of Pennsylvania Perelman School of Medicine, Philadelphia; Departments of Neurosurgery (J.R.H., A.R.F., G.T.M.), Epidemiology and Biostatistics (W.J.B., K.Y.), Neurology (K.Y., J.K., R.C.G.), Psychiatry (K.Y.), and Radiology and Biomedical Imaging (E.Y.), University of California San Francisco; Department of Neurosurgery (L.N.), Medical College of Wisconsin, Madison; Departments of Neurological Surgery (J.K.B., N.T.) and Biostatistics (N.T.), University of Washington, Seattle; and Biostatistics Research Center (S.J.), Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla.
| | | |
Collapse
|
3
|
Ferris LM, Kontos AP, Eagle SR, Elbin RJ, Collins MW, Mucha A, McAllister TW, Broglio SP, McCrea M, Pasquina PF, Port NL. Utility of VOMS, SCAT3, and ImPACT Baseline Evaluations for Acute Concussion Identification in Collegiate Athletes: Findings From the NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium. Am J Sports Med 2022; 50:1106-1119. [PMID: 35179972 DOI: 10.1177/03635465211072261] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
BACKGROUND The Vestibular/Ocular-Motor Screening (VOMS) is a valuable component of acute (<72 hours) sports-related concussion (SRC) assessments and is increasingly used with the Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) instrument and the third edition of the Sport Concussion Assessment Tool (SCAT3). Research has suggested that VOMS acute postinjury scores are useful in identifying acute concussion. However, the utility of preseason baseline measurements to improve diagnostic accuracy remains ambiguous. To this end, there is a need to determine how reliable VOMS baseline assessments are across years and whether incorporating individuals' baseline performance improves diagnostic yield for acute concussions. PURPOSE To analyze VOMS, SCAT3, and ImPACT to evaluate the test-retest reliability of consecutive-year preseason baseline assessments to directly compare the diagnostic utility of these tools when incorporating baseline assessments versus using postinjury data alone to identify acute SRC. STUDY DESIGN Cohort study (diagnosis); Level of evidence, 2. METHODS Preseason and postinjury VOMS, SCAT3, ImPACT Post-Concussion Symptom Scale (PCSS), and ImPACT composite scores were analyzed for 3958 preseason (47.7% female) and 496 acute (≤48 hours) SRC (37.5% female) collegiate athlete evaluations in the National Collegiate Athletic Association-Department of Defense Concussion Assessment Research and Education Consortium. Descriptive statistics, Kolmogorov-Smirnov significance, and Cohen d effect size were calculated. Consecutive-year baseline reliability was evaluated for a subset of 447 athlete encounters using Pearson r, Cohen κ, Cohen d, and 2-way mixed intraclass correlation coefficients (ICCs). Wilcoxon signed rank tests were used to determine the statistical significance between population performances, and the 90% reliable change index (RCI) was calculated from the test-retest results. Preseason to postinjury change scores were then calculated from each tool's RCI. Finally, receiver operating characteristic (ROC) curve analyses were conducted, and DeLong method was used to compare the area under the curve (AUC) of raw postinjury scores versus change scores from preseason baseline assessments. Potential effects of sex, medical history (learning disorders or attention-deficit/hyperactivity disorder), and outlier data were also explored. RESULTS Effect sizes were large, and overall predictive utilities were clinically useful for postinjury VOMS Total (d = 2.44; AUC = 0.85), the SCAT3 Symptom Evaluation total severity score (d = 1.74; AUC = 0.82), and the ImPACT PCSS total severity score (d = 1.67; AUC = 0.80). Comparatively, effect sizes were small and predictive utilities were poor for Standardized Assessment of Concussion (SAC), modified Balance Error Scoring System (mBESS), and all ImPACT composites (d = 0.11-0.46; AUC = 0.48-0.59). Preseason baseline test-retest reliability was poor to moderate (r = 0.23-0.52; κ = 0.32-0.36; ICC = 0.36-0.68) for all assessments except ImPACT Visual Motion Sensitivity (r = 0.73; ICC = 0.85). Incorporating baseline scores for VOMS Total, SCAT3 (Symptom Evaluation, SAC, mBESS), ImPACT PCSS, or ImPACT composites did not significantly improve AUCs. CONCLUSION VOMS Total and symptom severity (SCAT3, PCSS) total scores had large effect sizes and clinically useful AUCs for identifying acute concussion. However, all tools demonstrated high within-patient test-retest variability, resulting in poor reliability. The findings in this sample of collegiate athletes suggest that incorporating baseline assessments does not significantly increase diagnostic yield for acute concussion.
Collapse
Affiliation(s)
- Lyndsey M Ferris
- Indiana University School of Optometry, Bloomington, Indiana, USA
| | | | - Shawn R Eagle
- University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - R J Elbin
- University of Arkansas, Fayetteville, Arkansas, USA
| | | | - Anne Mucha
- UPMC Centers for Rehab Services, Pittsburg, Pennsylvania, USA
| | - Thomas W McAllister
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Steven P Broglio
- Michigan Concussion Center, University of Michigan, Ann Arbor, Michigan, USA
| | - Michael McCrea
- Neurosurgery, Medical College of Wisconsin, Millwauke, Wisconsin, USA
| | - Paul F Pasquina
- Department of Physical Medicine and Rehabilitation, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Nicholas L Port
- Indiana University School of Optometry, Bloomington, Indiana, USA
| |
Collapse
|