Validity of an Automated Balance Error Scoring System

Validity and Reliability of Methods to Assess Movement Deficiencies Following Concussion: A COSMIN Systematic Review

BACKGROUND

There is an increased risk of subsequent concussion and musculoskeletal injury upon return to play following a sports-related concussion. Whilst there are numerous assessments available for clinicians for diagnosis and during return to play following concussion, many may lack the ability to detect these subclinical changes in function. Currently, there is no consensus or collated sources on the reliability, validity and feasibility of these assessments, which makes it difficult for clinicians and practitioners to select the most appropriate assessment for their needs.

OBJECTIVES

This systematic review aims to (1) consolidate the reliability and validity of motor function assessments across the time course of concussion management and (2) summarise their feasibility for clinicians and other end-users.

METHODS

A systematic search of five databases was conducted. Eligible studies were: (1) original research; (2) full-text English language; (3) peer-reviewed with level III evidence or higher; (4) assessed the validity of lower-limb motor assessments used to diagnose or determine readiness for athletes or military personnel who had sustained a concussion or; (5) assessed the test-retest reliability of lower-limb motor assessments used for concussion management amongst healthy athletes. Acceptable lower-limb motor assessments were dichotomised into instrumented and non-instrumented and then classified into static (stable around a fixed point), dynamic (movement around a fixed point), gait, and other categories. Each study was assessed using the COSMIN checklist to establish methodological and measurement quality.

RESULTS

A total of 1270 records were identified, with 637 duplicates removed. Titles and abstracts of 633 records were analysed, with 158 being retained for full-text review. A total of 67 records were included in this review; 37 records assessed reliability, and 35 records assessed the validity of lower-limb motor assessments. There were 42 different assessments included in the review, with 43% being non-instrumented, subjective assessments. Consistent evidence supported the use of instrumented assessments over non-instrumented, with gait-based assessments demonstrating sufficient reliability and validity compared to static or dynamic assessments.

CONCLUSION

These findings suggest that instrumented, gait-based assessments should be prioritised over static or dynamic balance assessments. The use of laboratory equipment (i.e. 3D motion capture, pressure sensitive walkways) on average exhibited sufficient reliability and validity, yet demonstrate poor feasibility. Further high-quality studies evaluating the reliability and validity of more readily available devices (i.e. inertial measurement units) are needed to fill the gap in current concussion management protocols. Practitioners can use this resource to understand the accuracy and precision of the assessments they have at their disposal to make informed decisions regarding the management of concussion.

TRAIL REGISTRATION

This systematic review was registered on PROSPERO (reg no. CRD42021256298).

The Relationship Between Human-rated Errors and Tablet-based Postural Sway During the Balance Error Scoring System in Military Cadets

BACKGROUND

The Balance Error Scoring System (BESS) is commonly accepted as a valid measure of postural stability. However, reliability values have varied, and subtle changes undetectable with the human eye may exist postinjury. The inertial measurement unit in commercially available tablets has been used to quantify postural sway (instrumented Balance Error Scoring System [iBESS] volume). However, iBESS has not been validated in a military population, and the stability of the tests beyond 1 week is unknown.

HYPOTHESIS

iBESS volume is capable of objectively measuring postural sway during the traditional BESS.

STUDY DESIGN

Prospective repeated-measures study.

LEVEL OF EVIDENCE

Level 3.

METHODS

Eighty-three cadets (40.96% women; age 20.0 ± 1.44 years; height 68.7 ± 4.1 inches; weight 166.7 ± 30.2 lb) with no history of concussion or lower extremity injury agreed to participate. All participants completed the BESS at baseline and 6 months post baseline. During testing, a tablet equipped with an inertial measurement unit was positioned on the participant's sacrum to capture postural sway.

RESULTS

Moderate to strong correlations were exhibited between baseline measurements for single-limb (SL)-firm (r = 0.84; P < 0.01), tandem (TAN)-firm (r = 0.85; P < 0.01), double-limb (DL)-foam (r = 0.50; P < 0.01), SL-foam (r = 0.59; P < 0.01), and TAN-foam (r = 0.79; P < 0.01). Balance improved significantly at 6 months for SL-firm human-rated errors (Effect Size [ES] = 0.32) and for SL-firm (ES = 0.38), DL-foam (ES = 0.21), and SL-foam iBESS volume (ES = 0.35). Moderate to strong correlations were exhibited between human-rated and iBESS change scores for SL-firm (r = 0.71; P < 0.01), TAN-firm (r = 0.75; P < 0.01), and TAN-foam (r = 0.71; P < 0.01), and a weak correlation was exhibited for DL-foam (r = 0.29; P < 0.01) and SL-foam (r = 0.40; P < 0.01).

CONCLUSION

Moderate to strong correlations existed between human-rated BESS errors and iBESS volume at baseline and between change scores. In addition, iBESS volume may be more sensitive to balance changes than the human-rated BESS.

CLINICAL RELEVANCE

This evidence supports the use of iBESS volume as a valid measure of postural stability in military cadets. iBESS volume may provide clinicians with an objective and more sensitive measure of postural stability than the traditional human-rated BESS.

Level of Agreement Between Human-Rated and Instrumented Balance Error Scoring System Scores

Clinicians have used the Balance Error Scoring System (BESS) to quantify postural control for concussion management. However, the reliability of the human rated BESS has varied prompting the development of instrumented BESSs. A cross-sectional design was used to determine the level of agreement (LOA) between human rated and instrumented BESS scores. Sixty participants completed the BESS on video. An instrumented mat was used to quantify BESS errors while a live human rater simultaneously scored the BESS. A second human rated BESS performance via video. Bland-Altman LOA analyses evaluated agreement between scoring methods (Mat-Human, Mat-Video, Video-Live) for each stance. Mean biases between scores, for each stance, with 95% confidence intervals (95%CIs) were calculated. Agreement between scoring methods was not assessed for the Firm-Double-Limb stance because very few errors were recorded. Agreement between both human raters and the mat was poor based on mean bias estimates > ± 1 and/or wide 95%CIs for all stances including BESS-Total. Agreement between the human raters was better, having displayed consistently smaller mean bias estimates and tighter 95%CIs for all stances and BESS Total. As a result, human rated and instrumented BESS scores may not be comparable. One method should be used to measure BESS errors for consistency.