A patient’s recollection of pre-operative status is not accurate one year after arthroplasty of the hip or knee

Recall Bias in Client-Reported Outcomes in Canine Orthopaedic Patients Using Clinical Metrology Instruments

OBJECTIVE

 The aim of this study was to determine the accuracy of client recollection of their dogs' preconsultation status using clinical metrology instruments such as the Liverpool Osteoarthritis in Dogs (LOAD) and Canine Brief Pain Inventory (CBPI) questionnaires in dogs presenting to a referral orthopaedic clinic.

STUDY DESIGN

 This is a longitudinal prospective cohort study of client-owned dogs presenting for investigations of lameness (n = 217). LOAD and CBPI questionnaires were completed by the owners at the first consultation (T0). Owners were contacted at 2 (T1), 6 (T2), and 12 (T3) months and asked to recall their dogs' T0 status by completing another LOAD and CBPI questionnaire. The agreement between the T0 and recalled LOAD and CBPI scores was determined using the two-way mixed effects intraclass correlation coefficient (ICC). The Wilcoxon signed-rank test was used to determine the difference between scores.

RESULTS

 For the LOAD scores, there was moderate agreement between T0 and T1 (ICC: 0.64) and T0 and T2 (ICC: 0.53) scores and poor agreement between T0 and T3 (ICC: 0.496). For the CBPI Pain Severity Scores, there was poor agreement between T0 and all three subsequent time points (ICC < 0.5). For the CBPI Pain Interference Scores, there was moderate agreement between T0 and T1 (ICC: 0.57) and T2 (ICC: 0.56) scores and poor agreement between T0 and T3 (ICC: 0.43).

CONCLUSION

 The LOAD and CBPI questionnaires are subject to recall bias. Studies reporting retrospectively acquired CMI data should be interpreted with caution.

Do Patients Accurately Recall Their Preoperative Symptoms After Elective Orthopedic Procedures?

Patient-reported outcome measures are a frequent tool used to assess orthopedic surgical outcomes. However, recall bias is a potential limitation of these tools when used retrospectively, as they rely on patients to accurately recall their preoperative symptoms. A database search of Cochrane Library, PubMed, Medline Ovid, and Scopus until May 2021 was completed in duplicate by two reviewers. Studies considered eligible for inclusion were those which reported on patient recall bias associated with orthopedic surgery. The primary outcome of interest investigated was the accuracy of patient recollection of preoperative health status. Any factors that were identified as affecting patient recall were secondary outcomes of interest. Of the 4,065 studies initially screened, 20 studies with 3,454 patients were included in the final analysis. Overall, there were 2,371 (69%) knee and hip patients, 422 (12%) shoulder patients, 370 (11%) spine patients, 208 (6%) other upper extremity patients, and 83 (2%) foot and ankle patients. Out of the eight studies that evaluated patient recall within three months postoperatively, seven studies concluded that patient recall is accurate. Out of the 13 studies that evaluated patient recall beyond three months postoperatively, nine studies concluded that patient recall is inaccurate. The accuracy of patient recall of preoperative symptoms after elective orthopedic procedures is not reliable beyond three months postoperatively.

High recall bias in retrospective assessment of the pediatric International Knee Documentation Committee Questionnaire (Pedi-IKDC) in children with knee pathologies

PURPOSE

The increasing incidence of knee injuries among children is well known by sports physicians. Papers dealing with this topic have often collected patient-reported outcomes measures (PROMs) in a retrospective manner; this limitation could lead to a misinterpretation of the results, because pediatric patients might not remember their preoperative conditions adequately. This study aims to evaluate the reliability and the reproducibility of the IKDC pediatric score when administered retrospectively at a 12-month follow-up.

METHODS

From September 2018 and June 2019, all patients aged 7-18 scheduled for surgery due to different knee pathologies in a single center were considered eligible. Parents were contacted by phone for consent. An open-source platform was implemented to collect the responses: two surveys were created (Q1, Q2). They included general information and the Pedi-IKDC score. Q1 was completed prospectively, while Q2 was completed 12 months after surgery. The two questionnaires were identical, and patients were carefully advised to complete Q2 recalling their health status before surgery. ICC and the concordance correlation coefficient (ρ_c) were used to assess the reproducibility between the prospective and recalled scores.

RESULTS

Sixty-six patients responded to Q1 and Q2, and the mean age was 12.9 ± 2.2 years at Q1 and 14.1 ± 2.2 years at Q2. The mean time between Q1 and Q2 was 14.1 ± 2.1 months. Between prospective-IKDC and recall-IKDC, the ICC coefficient was "poor" at 0.32 (CI 0.09 to 0.5) and the ρ_c was "poor" at 0.4 (CI 0.29 to 0.51). Mean prospective-IKDC was 76.8 ± 23.52 mean recalled-IKDC was 60.4 ± 11.5 (P < 0.0001), while mean difference was -16.3 ± 2.09. Simple linear regression models showed that Δ-IKDC is independently associated with age at Q1 (R² = 0.2676; P0.0001)  and prospective-IKDC (R² = 0.653; P < 0.0001).

CONCLUSIONS

Retrospective collection of the Pedi-IKDC score is not reliable and has high recall bias. This should be avoided in children with knee conditions.

LEVEL OF EVIDENCE

III.

Recall Bias in Retrospective Assessment of Preoperative American Shoulder and Elbow Surgeons Scores After Reverse Total Shoulder Arthroplasty

INTRODUCTION

Although reverse total shoulder arthroplasty (RTSA) has been shown to be effective for the treatment of cuff tear arthropathy (CTA), the patient's inability to accurately recall their preoperative shoulder condition could skew their perception of the effectiveness of the procedure. Identifying patients who are susceptible to notable recall bias before surgery can help surgeons counsel patients regarding expectations after surgery. The purpose of this study was to evaluate whether patients who undergo RTSA are susceptible to recall bias and, if so, which factors are associated with poor recollection.

METHODS

Patients who underwent RTSA for CTA by the senior author between September 2016 and September 2018 were identified. All patients completed the American Shoulder and Elbow Surgeons (ASES scores) Standardized Assessment Form at the time of preoperative assessment. Patients were contacted at a minimum of 24 months after surgery to retrospectively assess their preoperative condition.

RESULTS

A total of 72 patients with a mean age of 72.2 ± 7.65 years completed a retrospective shoulder assessment at 28.3 ± 7.3 months postoperatively. Patient assessment of shoulder condition showed poor reliability (intraclass correlation coefficient = 0.453, confidence interval, 0.237-0.623). Greater preoperative shoulder ASES scores were associated with a greater difference between preoperative ASES scores and recall ASES scores (β = 0.275, P < 0.001).

CONCLUSION

Patients who undergo RTSA for CTA are susceptible to clinically significant recall bias. Patients with better preoperative condition recall worse preoperative shoulder conditions compared with patients with worse preoperative conditions and are susceptible to a higher degree of recall bias. This patient population should be identified preoperatively and have notable counseling before and after surgery to help them better understand their disease burden and what to expect after surgical intervention.

LEVEL OF EVIDENCE

III, diagnostic cohort study.

Recall Bias in Retrospective Assessment of Preoperative Patient-Reported American Shoulder and Elbow Surgeons Scores in Arthroscopic Rotator Cuff Repair Surgery

BACKGROUND

The gold-standard method for collecting patient-reported outcomes (PROs) is the prospective assessment of preoperative to postoperative change. However, this method is not always feasible because of unforeseen cases or emergencies, logistical and infrastructure barriers, and cost issues. In such cases, a retrospective approach serves as a potential alternative, but there are conflicting conclusions regarding the reliability of the recalled preoperative PROs after orthopaedic procedures.

PURPOSE

To assess the agreement between prospectively and retrospectively collected PROs for a common, low-risk procedure.

STUDY DESIGN

Cohort study (Diagnosis); Level of evidence, 3.

METHODS

Patients who underwent arthroscopic rotator cuff repair between May 2012 and September 2017 at the study institution were identified. All of the patients completed the American Shoulder and Elbow Surgeons (ASES) Standard Shoulder Assessment Form preoperatively at their preassessment appointment. Patients were then contacted in the postoperative period and asked to recall their preoperative condition while completing another ASES form.

RESULTS

A total of 84 patients completed the telephone survey and were included in this analysis (mean age, 57.40 ± 9.96 years). The mean duration of time from onset of shoulder symptoms to surgery was 9.13 ± 9.08 months. The mean duration of time between surgery and recall ASES administration was 39.12 ± 17.37 months. The mean recall ASES score was significantly lower than the preoperative ASES score (30.69 ± 16.93 vs 51.42 ± 19.14; P < .001). There was poor test-retest reliability between preoperative ASES and recall ASES (intraclass correlation coefficient, 0.292; 95% CI, -0.07, 0.57; P = .068). Greater age at the time of recall, a shorter symptomatic period before surgery, and less severe preoperative shoulder dysfunction were associated with a greater difference between preoperative ASES and recall ASES.

CONCLUSION

Retrospectively reported PROs are subject to significant recall bias. Recalled PROs were almost always lower than their prospectively recorded counterparts. Recalled PROs are more likely to be accurate when reported by younger patients, those with a longer duration of symptoms, and those with more severe preoperative conditions.

Patients Undergoing Shoulder Stabilization Procedures Do Not Accurately Recall Their Preoperative Symptoms at Short- to Midterm Follow-up

Background:

A patient’s ability to recall symptoms is poor in some elderly populations, but we considered that the recall of younger patients may be more accurate. The accuracy of recall in younger patients after surgery has not been reported to date.

Purpose:

To assess younger patients’ abilities to recall their preoperative symptoms after having undergone shoulder stabilization surgery. We used 2 disease-specific, patient-reported outcome measures (PROMs)—the Western Ontario Shoulder Instability Index (WOSI) and the Melbourne Instability Shoulder Score (MISS)—at a period of up to 2 years postoperatively.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Methods:

Participants (N = 119) were stratified into 2 groups: early recall (at 6-8 months postoperatively; n = 58) and late recall (at 9-24 months postoperatively; n = 61). All patients completed the PROMs with instructions to recall preoperative function. The mean and absolute differences between the preoperative scores and recalled scores for each PROM were compared using paired t tests. Correlations between the actual and recalled scores of the subsections for each PROM were calculated using an intraclass correlation coefficient (ICC). The number of individuals who recalled within the minimal detectable change (MDC) of each PROM was calculated.

Results:

Comparison between the means of the actual and recalled preoperative scores for both groups did not demonstrate significant differences (early recall differences, MISS 1.05 and WOSI –38.64; late recall differences, MISS –0.25 and WOSI –24.02). Evaluation of the absolute difference, however, revealed a significant difference between actual and recalled scores for both the late and early groups (early recall absolute differences, MISS 12.26 and WOSI 216.71; late recall absolute differences, MISS 12.84 and WOSI 290.08). Average absolute differences were above the MDC scores of both PROMs at both time points. Subsections of each PROM demonstrated weak to moderate correlations between actual and recalled scores (ICC range, 0.17-0.61). Total scores for the PROMs reached moderate agreement between actual and recalled scores.

Conclusion:

Individual recall after shoulder instability surgery was not accurate. However, the mean recalled PROM scores of each group were not significantly different from the actual scores collected preoperatively, and recall did not deteriorate significantly over 2 years. This suggests that recall of the individual, even in this younger group, cannot be considered accurate for research purposes.

The Specific AC Score (SACS): a new and validated method of assessment of isolated acromioclavicular joint pathology

BACKGROUND

Acromioclavicular (AC) joint (ACJ) pathology is a common cause of shoulder dysfunction, and treatment recommendations vary. When the efficacy of treatment is evaluated, the ability to measure outcomes specific to the population is essential. The aim of the current research was to develop and validate a specific ACJ questionnaire.

METHODS

Items for the "Specific AC Score" (SACS) were generated through the use of an expert panel, existing questionnaires, and patient feedback. Preliminary data analysis identified redundancy of items resulting in the questionnaire being refined. The final SACS was evaluated in 125 patients requiring surgical intervention of the ACJ. Internal consistency (the Cronbach α and corrected item-total correlation), content validity, criterion validity, responsiveness, and test-retest reliability (intraclass correlation coefficient) were examined and compared with the Shoulder Pain and Disability Index, Oxford Shoulder Score, and American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form. The minimum detectable change score was calculated.

RESULTS

The Cronbach α for the total scale preoperatively and postoperatively was high (preoperatively = 0.91, postoperatively = 0.93). All 3 domains (Pain, Function, Quality of Life) demonstrated acceptable internal consistency (α > 0.70), and the correlation between items in each domain was satisfactory. The responsiveness was excellent (effect size, -2.32; standard response mean, -1.85) and was higher than the other general shoulder questionnaires. There were no relevant floor or ceiling effects. Reliability was high (intraclass correlation coefficient, 0.89) and the minimum detectable change was 6.5 points.

DISCUSSION

This new ACJ-specific questionnaire has been robustly developed, has good measurement properties, and has excellent responsiveness. The SACS is recommended for measuring outcomes in ACJ patients.

Recall of preoperative Oxford Hip and Knee Scores one year after arthroplasty is an alternative and reliable technique when used for a cohort of patients

Objectives

The primary aim of this study was to assess the reproducibility of the recalled preoperative Oxford Hip Score (OHS) and Oxford Knee Score (OKS) one year following arthroplasty for a cohort of patients. The secondary aim was to assess the reliability of a patient's recollection of their own preoperative OHS and OKS one year following surgery.

Methods

A total of 335 patients (mean age 72.5; 22 to 92; 53.7% female) undergoing total hip arthroplasty (n = 178) and total knee arthroplasty (n = 157) were prospectively assessed. Patients undergoing hip and knee arthroplasty completed an OHS or OKS, respectively, preoperatively and were asked to recall their preoperative condition while completing the same score one year after surgery.

Results

A mean difference of 0.04 points (95% confidence intervals (CI) -15.64 to 15.72, p = 0.97) between the actual and the recalled OHS was observed. The mean difference in the OKS was 1.59 points (95% CI -11.57 to 14.75, p = 0.10). There was excellent reliability for the 'average measures' intra-class correlation for both the OHS (r = 0.802) and the OKS (r = 0.772). However, this reliability was diminished for the individuals OHS (r = 0.670) and OKS (r = 0.629) using single measures intra-class correlation. Bland-Altman plots demonstrated wide variation in the individual patient's ability to recall their preoperative score (95% CI ± 16 for OHS, 95% CI ± 13 for OKS).

Conclusion

Prospective preoperative collection of OHS and OKS remains the benchmark. Using recalled scores one year following hip and knee arthroplasty is an alternative when used to assess a cohort of patients. However, the recall of an individual patient's preoperative score should not be relied upon due to the diminished reliability and wide CI.Cite this article: T. F. M. Yeoman, N. D. Clement, D. Macdonald, M. Moran. Recall of preoperative Oxford Hip and Knee Scores one year after arthroplasty is an alternative and reliable technique when used for a cohort of patients. Bone Joint Res 2018;7:351-356. DOI: 10.1302/2046-3758.75.BJR-2017-0259.R1.

Patients recall worse preoperative pain after shoulder arthroplasty than originally reported: a study of recall accuracy using the American Shoulder and Elbow Surgeons score

BACKGROUND

Patient-reported outcome measures (PROMs) are valuable tools for quantifying outcomes of orthopedic surgery. However, when baseline scores are not obtained, there is considerable controversy about whether PROMs can be administered retrospectively for patients to recall their preoperative state. We investigated the accuracy of patient recall after total shoulder arthroplasty (TSA) using the American Shoulder and Elbow Surgeons (ASES) assessment score.

METHODS

Recalled ASES scores were collected postoperatively at 6 weeks, 3 months, 6 months, and 12 months from 169 patients who previously completed baseline scores before TSA. The ASES total score was divided into its two subcomponents: functional ability and visual analog scale (VAS) for pain. We compared preoperative and recalled scores for each subcomponent and the total ASES score.

RESULTS

Recalled ASES function scores were comparable to corresponding preoperative scores across all time points (analysis of variance, P = .21), but recalled VAS pain was significantly higher at all time points beyond 6 weeks after surgery (P = .0001 at 3 months; P = .005 at 6 months; and P = .001 at 12 months). As a result, the ASES total score was only comparable at 6 weeks after surgery (P = .39) and differed at all time points thereafter.

CONCLUSION

Patients are able to recall preoperative function with considerable accuracy for up to 12 months after TSA. However, beyond 6 weeks postoperatively, patients recall having worse pain than they originally reported, and recalled ASES total scores are unreliable as a result.