How Important Is Randomization of Swallows During Kinematic Analyses of Swallow Function?

Scoring the Penetration-Aspiration Scale (PAS) in Two Conditions: A Reliability Study

A widely applied metric for identifying airway invasion events is the Penetration-Aspiration Scale (PAS). PAS scores are often included as primary outcome measures in clinical interventional studies, applied to characterize airway protection in a particular disease, used to establish a normal referent for control group comparisons without dysphagia, and as determinants or predictors of clinical outcomes. Despite the widespread use of the PAS, there is variability in scoring condition. One common method used in research studies includes rater scores applied to each single swallow that occurred during a modified barium swallow study (MBSS) of the same patient. A second common method includes raters scoring single swallow segments that have been spliced from full MBSS from different patients. These single swallow segments are then randomly distributed and the rater is blinded to all swallows that occurred during that patient MBSS. The potential effects of different scoring conditions on rater reliability and score accuracy have not been studied and may have high relevance for the conclusion drawn from the result. The primary aim of this investigation is to determine the impact of two scoring conditions on rater reliability and score accuracy: 1. Contextual, unblinded scoring condition and 2. Randomized, blinded condition. Results of the present study show that no statistically significant differences in PAS rater reliability and score accuracy were found between the two scoring conditions. If findings from this pilot study are reproduced in larger sample sizes, the time and intensity involved in splicing and randomizing MBSS for scoring may not be necessary.

Reliability of the Penetration-Aspiration Scale and Temporal and Clearance Measures in Poststroke Dysphagia: Videofluoroscopic Analysis From the Swallowing Treatment using Electrical Pharyngeal Stimulation Trial

PURPOSE

Information on reliability of outcome measures used to assess the effectiveness of interventions in dysphagia rehabilitation is lacking, particularly when used by different research groups. Here, we report on reliability of the penetration-aspiration scale (PAS) and temporal and clearance measures, determined using videofluoroscopy.

METHOD

Secondary analysis used videofluoroscopies from the Swallowing Treatment using Electrical Pharyngeal Stimulation trial in subacute stroke. PAS scores (719 scores from 18 participants) were evaluated and compared to the original PAS scores from the trial. Five conditions were assessed, including reliability for every swallow and overall mean of the worst PAS score. Operational rules for assessing temporal and clearance measures were also developed using the same data, and reliability of these rules was assessed. Reliability of component-level and derivative-level scores was assessed using the intraclass correlation coefficient (ICC) and weighted kappa.

RESULTS

Image quality was variable. Interrater reliability for the overall mean of the worst PAS score was excellent (ICC = .914, 95% confidence interval [CI] [.853, .951]) but moderate for every swallow in the bolus (ICC = .743, 95% CI [.708, .775]). Intrarater reliability for PAS was excellent (all conditions). Excellent reliability (both inter- and intrarater > .90) was seen for temporal measures of stage transition duration (ICC = .998, 95% CI [.993, .999] and ICC = .995, 95% CI [.987, .998], respectively) as well as initiation of laryngeal closure and pharyngeal transit time and all individual swallow events. Strong scores were obtained for some clearance measures; others were moderate or weak.

CONCLUSIONS

Interrater reliability for PAS is acceptable but depends on how the PAS scores are handled in the analysis. Interrater reliability for most temporal measures was high, although some measures required additional training. No clearance measures had excellent reliability.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19090088.