Reliability and Minimal Detectable Change for a Smartphone-Based Motor-Cognitive Assessment: Implications for Concussion Management

A Multifaceted Approach to Interpreting Reaction Time Deficits After Adolescent Concussion

CONTEXT

Reaction time (RT) is a critical element of return to participation (RTP), and impairments have been linked to subsequent injury after a concussion. Current RT assessments have limitations in clinical feasibility and in the identification of subtle deficits after concussion symptom resolution.

OBJECTIVES

To examine the utility of RT measurements (clinical drop stick, simple stimulus-response, single-task Stroop, and dual-task Stroop) to differentiate between adolescents with concussion and uninjured control individuals at initial assessment and RTP.

DESIGN

Prospective cohort study.

SETTING

A pediatric sports medicine center associated with a regional tertiary care hospital.

PATIENTS OR OTHER PARTICIPANTS

Twenty-seven adolescents with a concussion (mean age = 14.8 ± 2.1 years; 52% female; tested 7.0 ± 3.3 days postconcussion) and 21 uninjured control individuals (mean age = 15.5 ± 1.6 years; 48% female).

MAIN OUTCOME MEASURE(S)

Participants completed the Post-Concussion Symptoms Inventory (PCSI) and a battery of RT tests: clinical drop stick, simple stimulus-response, single-task Stroop, and dual-task Stroop.

RESULTS

The concussion group demonstrated slower clinical drop stick (β = 58.8; 95% CI = 29.2, 88.3; P < .001) and dual-task Stroop (β = 464.2; 95% CI = 318.4, 610.0; P < .001) RT measures at the initial assessment than the uninjured control group. At 1-month follow up, the concussion group displayed slower clinical drop stick (238.9 ± 25.9 versus 188.1 ± 21.7 milliseconds; P < .001; d = 2.10), single-task Stroop (1527.8 ± 204.5 versus 1319.8 ± 133.5 milliseconds; P = .001; d = 1.20), and dual-task Stroop (1549.9 ± 264.7 versus 1341.5 ± 114.7 milliseconds; P = .002; d = 1.04) RT than the control group, respectively, while symptom severity was similar between groups (7.4 ± 11.2 versus 5.3 ± 6.5; P = .44; d = 0.24). Classification accuracy and area under the curve (AUC) values were highest for the clinical drop stick (85.1% accuracy, AUC = 0.86, P < .001) and dual-task Stroop (87.2% accuracy, AUC = 0.92, P < .002) RT variables at initial evaluation.

CONCLUSIONS

Adolescents recovering from concussion may have initial RT deficits that persist despite symptom recovery. The clinical drop stick and dual-task Stroop RT measures demonstrated high clinical utility given high classification accuracy, sensitivity, and specificity to detect postconcussion RT deficits and may be considered for initial and RTP assessment.

Minimal Detectable Change for the ImPACT Subtests at Baseline

OBJECTIVE

To establish the minimal detectable change (MDC) of the subtests that comprise the composite scores from remotely administered Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) baselines.

METHOD

Remote ImPACT baseline data from 172 (male = 45, female = 127) National Collegiate Athletic Association Division I student-athletes from the 2020 and 2021 athletic preseasons were used to calculate the MDC at the 95%, 90%, and 80% confidence intervals (CIs) for all subtest scores used to generate the four core composite scores and the impulse control composite.

RESULTS

The MDCs for the verbal memory subtests at the 95% CI were 10.31 for word memory percent correct, 4.68 for symbol match total correct hidden, and 18.25 for three letters percentage correct. Visual memory subtest MDCs were 19.03 for design memory total percent correct and 4.90 for XO total correct memory. Visual motor speed subtest MDCs were 18.89 for XO total correct interference and 5.40 for three letters average counted correctly. Reaction time (RT) MDCs were 0.12 for XO average correct, 0.95 for symbol match average correct RT, and 0.28 for color match average correct. Impulse control MDCs were 5.97 for XO total incorrect and 1.15 for color match total commissions. One-way repeated measures MANOVA, repeated measures ANOVAs, and Wilcoxon signed-ranks test all suggested no significant difference between any subtests across two remote ImPACT baselines.

CONCLUSIONS

The ImPACT subtest scores did not significantly change between athletic seasons. Our study suggests the subtests be evaluated in conjunction with the composite scores to provide additional metrics for clinical interpretation.

Minimal Detectable Change for the ImPACT Test Administered Remotely

CONTEXT

Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is one of the most widely used computerized neurocognitive assessment batteries in athletics and serves as both a baseline and postinjury assessment. It has become increasingly popular to administer the ImPACT baseline test in an unsupervised remote environment; however, whether the lack of supervision affects the test-retest reliability is unknown.

OBJECTIVE

To establish the minimal detectable change (MDC) of composite scores from the ImPACT test when administered to National Collegiate Athletic Association Division I student-athletes in an unsupervised remote environment before 2 consecutive athletic seasons.

DESIGN

Cross-sectional study.

SETTING

Participants were provided with a unique link and detailed written instructions on how to complete the ImPACT test at home.

PATIENTS OR OTHER PARTICIPANTS

Division I student-athletes.

MAIN OUTCOME MEASURE(S)

Remote baseline ImPACT results from the 2020-2021 and 2021-2022 athletic seasons were analyzed. The MDC was calculated at the 95%, 90%, and 80% CIs for each of the ImPACT composite scores as well as the average and SD.

RESULTS

The MDC at the 95% CI was 18.6 for the verbal memory composite score, 24.44 for visual memory, 8.76 for visual motor, 0.14 for reaction time, and 6.13 for impulse control. One-way repeated-measures multivariate analysis of variance, repeated-measures analysis of variance, and Wilcoxon signed ranks tests suggested no difference in the composite scores and impulse control between time points.

CONCLUSIONS

The ImPACT composite scores and impulse control did not change between the 2 remote testing time points when administered approximately 1 year apart. Our study suggests that the MDC serves as a clinician's guide for evaluating changes in ImPACT baseline scores and in making clinical judgments on sport-related concussion when the test is administered at home.

Ankle Sprain History Does Not Significantly Alter Single- and Dual-Task Spatiotemporal Gait Mechanics

CONTEXT

Single- and dual-task walking gait assessments have been used to identify persistent movement and cognitive dysfunction among athletes with concussions. However, it is unclear whether previous ankle sprain injuries confound these outcomes during baseline testing. The purpose of this study was to determine the effects of (1) ankle sprain history and (2) time since prior ankle sprain injury on single- and dual-task spatiotemporal gait outcomes and cognitive measures.

DESIGN

Cross-sectional study.

METHODS

We assessed 60 college Division-I athletes (31 with ankle sprain history; 13 females and 18 males, 19.3 [0.8] y; 29 with no ankle sprain history, 14 females and 15 males, 19.7 [0.9] y) who completed injury history forms and underwent concussion baseline testing. Athletes completed single- and dual-task gait assessments by walking back and forth along an 8-m walkway for 40 seconds. Athletes wore a smartphone with an associated mobile application on their lumbar spine to record spatiotemporal gait parameters and dual-task cognitive performance. Separate multivariate analyses of variance were used to assess the effects of ankle sprain injury history on spatiotemporal measures, gait variability, and cognitive performance. We performed a multivariate regression subanalysis on athletes who reported time since injury (n = 23) to assess temporal effects on gait and cognitive performance.

RESULTS

Athletes with and without a history of ankle sprains had comparable spatiotemporal and gait variability outcomes during single- (P = .42; P = .13) and dual-task (P = .75; P = .55) conditions. Additionally, ankle sprain injury history did not significantly influence cognitive performance (P = .35). Finally, time since ankle sprain did not significantly affect single- (P = .75) and dual-task gait (P = .69), nor cognitive performance (P = .19).

CONCLUSIONS

Ankle sprain injury history did not significantly alter spatiotemporal gait outcomes nor cognitive performance during this common clinical assessment. Future studies may consider including athletes with ankle sprain injury history during concussion assessments.

Validity and Reliability of Methods to Assess Movement Deficiencies Following Concussion: A COSMIN Systematic Review

BACKGROUND

There is an increased risk of subsequent concussion and musculoskeletal injury upon return to play following a sports-related concussion. Whilst there are numerous assessments available for clinicians for diagnosis and during return to play following concussion, many may lack the ability to detect these subclinical changes in function. Currently, there is no consensus or collated sources on the reliability, validity and feasibility of these assessments, which makes it difficult for clinicians and practitioners to select the most appropriate assessment for their needs.

OBJECTIVES

This systematic review aims to (1) consolidate the reliability and validity of motor function assessments across the time course of concussion management and (2) summarise their feasibility for clinicians and other end-users.

METHODS

A systematic search of five databases was conducted. Eligible studies were: (1) original research; (2) full-text English language; (3) peer-reviewed with level III evidence or higher; (4) assessed the validity of lower-limb motor assessments used to diagnose or determine readiness for athletes or military personnel who had sustained a concussion or; (5) assessed the test-retest reliability of lower-limb motor assessments used for concussion management amongst healthy athletes. Acceptable lower-limb motor assessments were dichotomised into instrumented and non-instrumented and then classified into static (stable around a fixed point), dynamic (movement around a fixed point), gait, and other categories. Each study was assessed using the COSMIN checklist to establish methodological and measurement quality.

RESULTS

A total of 1270 records were identified, with 637 duplicates removed. Titles and abstracts of 633 records were analysed, with 158 being retained for full-text review. A total of 67 records were included in this review; 37 records assessed reliability, and 35 records assessed the validity of lower-limb motor assessments. There were 42 different assessments included in the review, with 43% being non-instrumented, subjective assessments. Consistent evidence supported the use of instrumented assessments over non-instrumented, with gait-based assessments demonstrating sufficient reliability and validity compared to static or dynamic assessments.

CONCLUSION

These findings suggest that instrumented, gait-based assessments should be prioritised over static or dynamic balance assessments. The use of laboratory equipment (i.e. 3D motion capture, pressure sensitive walkways) on average exhibited sufficient reliability and validity, yet demonstrate poor feasibility. Further high-quality studies evaluating the reliability and validity of more readily available devices (i.e. inertial measurement units) are needed to fill the gap in current concussion management protocols. Practitioners can use this resource to understand the accuracy and precision of the assessments they have at their disposal to make informed decisions regarding the management of concussion.

TRAIL REGISTRATION

This systematic review was registered on PROSPERO (reg no. CRD42021256298).

Validity and reliability of the Apple Health app on iPhone for measuring gait parameters in children, adults, and seniors

This study assessed the concurrent validity and test-retest-reliability of the Apple Health app on iPhone for measuring gait parameters in different age groups. Twenty-seven children, 28 adults and 28 seniors equipped with an iPhone completed a 6-min walk test (6MWT). Gait speed (GS), step length (SL), and double support time (DST) were extracted from the gait recordings of the Health app. Gait parameters were simultaneously collected with an inertial sensors system (APDM Mobility Lab) to assess concurrent validity. Test-retest reliability was assessed via a second iPhone-instrumented 6MWT 1 week later. Agreement of the Health App with the APDM Mobility Lab was good for GS in all age groups and for SL in adults/seniors, but poor to moderate for DST in all age groups and for SL in children. Consistency between repeated measurements was good to excellent for all gait parameters in adults/seniors, and moderate to good for GS and DST but poor for SL in children. The Health app on iPhone is reliable and valid for measuring GS and SL in adults and seniors. Careful interpretation is required when using the Health app in children and when measuring DST in general, as both have shown limited validity and/or reliability.

A novel smartphone application is reliable for repeat administration and comparable to the Tekscan Strideway for spatiotemporal gait

Smartphone applications are increasingly being used to measure gait due to their portability and cost-effectiveness. Important reliability metrics are not available for most of these devices. The purpose of this article was to evaluate the test-retest reliability and concurrent validity of spatiotemporal gait using the novel Gait Analyzer smartphone application compared to the Tekscan Strideway. Healthy participants (n=23) completed 12 trials of 10-meter walking, at two separate time points, using Gait Analyzer and while walking across the Tekscan Strideway. The results suggest excellent test-retest reliability for the Gait Analyzer and good test-retest reliability for the Tekscan Strideway for both velocity and cadence. At both time points, these devices were moderately to strongly correlated to one another for both velocity and cadence. These data suggest that the Gait Analyzer and Tekscan Strideway are reliable over time and can comparably calculate velocity and cadence.