Applying the Generalizability Theory to Identify the Sources of Validity Evidence for the Quality of Communication Questionnaire

Validation of a method to assess the severity of medication administration errors in Brazil

INTRODUCTION

Medication errors are frequent and have high economic and social impacts; however, some medication errors are more likely to result in harm than others. Therefore, it is critical to determine their severity. Various tools exist to measure and classify the harm associated with medication errors; although, few have been validated internationally.

METHODS

We validated an existing method for assessing the potential severity of medication administration errors (MAEs) in Brazil. Thirty healthcare professionals (doctors, nurses and pharmacists) from Brazil were invited to score 50 cases of MAEs as in the original UK study, regarding their potential harm to the patient, on a scale from 0 to 10. Sixteen cases with known harmful outcomes were included to assess the validity of the scoring. To assess test-retest reliability, 10 cases (of the 50) were scored twice. Potential sources of variability in scoring were evaluated, including the occasion on which the scores were given, the scorers, their profession and the interactions among these variables. Data were analysed using generalisability theory. A G coefficient of 0.8 or more was considered reliable, and a Bland-Altman analysis was used to assess test-retest reliability.

RESULTS

To obtain a generalisability coefficient of 0.8, a minimum of three judges would need to score each case with their mean score used as an indicator of severity. The method also appeared to be valid, as the judges' assessments were largely in line with the outcomes of the 16 cases with known outcomes. The Bland-Altman analysis showed that the distribution was homogeneous above and below the mean difference for doctors, pharmacists and nurses.

CONCLUSION

The results of this study demonstrate the reliability and validity of an existing method of scoring the severity of MAEs for use in the Brazilian health system.