Vaichinger AM, Shields MN, Morrey ME, O'Driscoll SW. Prospective Blinded Evaluation of Patient-Physician Agreement Using the Summary Outcome Determination (SOD) Score.
Mayo Clin Proc 2019;
94:1231-1241. [PMID:
31248694 DOI:
10.1016/j.mayocp.2018.10.025]
[Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2018] [Revised: 09/20/2018] [Accepted: 10/03/2018] [Indexed: 01/30/2023]
Abstract
OBJECTIVE
To determine whether the Summary Outcome Determination (SOD) score demonstrates a high level of physician-patient agreement in a prospective setting with multiple raters.
PATIENTS AND METHODS
For this study, 100 patients who were being evaluated at various intervals following shoulder or elbow surgery were prospectively enrolled from May 30, 2017, through August 31, 2017. The patients' attending physicians and a member of their team (physician assistant, resident, fellow, medical student) assigned categorical and numerical SOD scores while blinded to the scores given by each other. All scores were analyzed among raters, assessing internal consistencies, agreement, and reliability.
RESULTS
The mean follow-up (interval between surgery and completion of the survey) was 31 months, with a range of 1 to 220 months. The intraclass correlation coefficient for patient and physician numerical scores was excellent at 0.82. The weighted κ value for categorical scores was 0.64. Bland-Altman analysis revealed low average discrepancy at 0.6 with a 95% CI of -3.3 to 4.5. The Cronbach α value was 0.94, indicating strong internal validity. The categorical physician-patient agreement occurred within one category 96% of the time.
CONCLUSION
This study found that the SOD score has strong agreement with excellent intraclass correlation coefficient and weighted κ values, indicating substantial agreement, reproducibility (shown by low average error), and strong internal validity. With promising results in the prospective setting, the SOD score was found to be an easy to use outcome measure with reliable agreement between patient and physician. This score has potential to be a metric revealing the "value" of a specific surgical intervention.
Collapse