The effect of informed consent on results of a standard upper extremity intake questionnaire

Defining and evaluating the Hawthorne effect in primary care, a systematic review and meta-analysis

In 2015, we conducted a randomized controlled trial (RCT) in primary care to evaluate if posters and pamphlets dispensed in general practice waiting rooms enhanced vaccination uptake for seasonal influenza. Unexpectedly, vaccination uptake rose in both arms of the RCT whereas public health data indicated a decrease. We wondered if the design of the trial had led to a Hawthorne effect (HE). Searching the literature, we noticed that the definition of the HE was unclear if stated. Our objectives were to refine a definition of the HE for primary care, to evaluate its size, and to draw consequences for primary care research. We designed a Preferred Reporting Items for Systematic reviews and Meta-Analyses review and meta-analysis between January 2012 and March 2022. We included original reports defining the HE and reports measuring it without setting limitations. Definitions of the HE were collected and summarized. Main published outcomes were extracted and measures were analyzed to evaluate odds ratios (ORs) in primary care. The search led to 180 records, reduced on review to 74 for definition and 15 for quantification. Our definition of HE is "an aware or unconscious complex behavior change in a study environment, related to the complex interaction of four biases affecting the study subjects and investigators: selection bias, commitment and congruence bias, conformity and social desirability bias and observation and measurement bias." Its size varies in time and depends on the education and professional position of the investigators and subjects, the study environment, and the outcome. There are overlap areas between the HE, placebo effect, and regression to the mean. In binary outcomes, the overall OR of the HE computed in primary care was 1.41 (95% CI: [1.13; 1.75]; I 2 = 97%), but the significance of the HE disappears in well-designed studies. We conclude that the HE results from a complex system of interacting phenomena and appears to some degree in all experimental research, but its size can considerably be reduced by refining study designs.

Patient and Surgeon Ratings of Patient Involvement in Decision-Making Are Not Aligned

BACKGROUND

Specialists want to guide patients toward making informed treatment choices consistent with what matters most to them (their values). One measure of this goal is alignment between patient and surgeon-perceived involvement in decision-making. We performed a cross-sectional survey of patients presenting for musculoskeletal specialty care to determine agreement between patients and surgeons regarding patient involvement in shared decision-making. We also tested (1) factors associated with specialist-perceived involvement, accounting for the patient's perceived involvement in decision-making, and (2) factors associated with patient perception of involvement in decision-making, accounting for ratings of preferred involvement.

METHODS

In this cross-sectional survey, 136 patients seeking musculoskeletal care for conditions involving the upper or lower extremities rated their preferred level of involvement in decision-making (Control Preferences Scale) before the visit and their perceived level of involvement (Modified Control Preferences Scale) after the visit. Participants also completed measures of symptoms of depression and pain self-efficacy. After the visit, the surgeons rated their perception of the patient's involvement in decision-making (Modified Control Preferences Scale).

RESULTS

There was poor agreement between patients and surgeons regarding the extent of patient participation in decision-making (ICC = 0.11). The median difference was 1 point on a 5-point Likert scale (interquartile range: 0 to 1). Accounting for demographic characteristics and personal factors in multivariable analysis, specialists rated patients who did not have a high school diploma as having less involvement in decision-making. Specialist-perceived patient involvement in decision-making was not related to patient-perceived involvement. The only factor associated with higher patient-rated involvement was higher patient-preferred involvement (OR = 3.9; 95% CI = 2.6 to 5.8; p < 0.001).

CONCLUSIONS

The observation that surgeons misperceive patient participation in decision-making emphasizes the need for strategies to ensure patient participation, such as methods to help patients gain awareness of what matters most to them (their values), clinician checklists for identification and reorientation of common misinterpretations of symptoms, and decision aids or motivational interviewing tools that can help to ensure that patient choices are consistent with their values and are unhindered by misconceptions.

A Comparison of PROMIS Physical Function and Pain Interference Scores in Patients With Carpal Tunnel Syndrome: Research Collection Versus Routine Clinical Collection

Background: There is a concern that patients may answer patient-reported outcome (PRO) questionnaires differently depending on the purpose-clinical care or research (eg, "Hawthorne effect"). We sought to determine whether Patient-Reported Outcomes Management Information System (PROMIS) scores differ at the same clinic visit based on whether a patient was completing the PRO tool for study or clinical care purposes. Methods: Patients presenting to one surgeon at an academic medical center hand clinic were asked to complete PROMIS Physical Function (PF) and Pain Interference (PI) questionnaires as part of routine care. Those diagnosed with carpal tunnel syndrome from February 2015 to April 2017 were then asked to participate in a clinical research project, which had them complete PROMIS PF and PROMIS PI again. Data from those who completed both routine and research PROs at each visit were compared. Between the 2 settings, test-retest reliability was determined using Pearson correlation coefficients (r), and internal consistency was evaluated using Cronbach α. Results: A total of 128 unique office visits representing 67 patients fit our inclusion criteria. There was a strong correlation between PROMIS PF and PI in the research and patient care setting (PF: r = 0.82, P < .01; PI: r = 0.83, P < .01). Both domains had a Cronbach α of 0.90. The PROMIS PF scores were not different between the 2 groups (P = .19), but the PROMIS PI scores were slightly different (P < .01). Conclusions: Patients appear to be consistent when completing PROMIS for both clinical care and research, supporting the idea that data obtained in either setting are generalizable and appropriate for research purposes.

Hawthorne effect on surgical studies

BACKGROUND

The Hawthorne effect or 'observer effect' describes a change in normal behaviour when individuals are aware they are being observed. This may have an impact on effect estimates in clinical trials. The purpose of this study was to determine if the Hawthorne effect had been recorded as a risk of bias in surgical studies.

METHODS

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses compliant literature search was conducted till March 2019. Eligible studies included those reporting or not reporting the Hawthorne effect in surgical studies from the following databases: MEDLINE, Embase, CINAHL, AMED, BNI, HMIC, PsycINFO, Web of Science, Cochrane Library, Google Scholar and OpenGrey. Two reviewers independently reviewed the papers, extracted data and appraised study methods using the Newcastle Ottawa Scale or the Cochrane risk of bias tool. Data were analysed descriptively.

RESULTS

A total of 842 papers were identified, of which 16 were eligible. Six (37%) observational studies were identified with the aim of measuring the Hawthorne effect on their outcome with five reporting that the Hawthorne effect was responsible for the improvements in outcomes and one reporting no change in outcome due to the Hawthorne effect. Ten (63%) studies were identified, of which eight used the Hawthorne effect as an explanation to improvements seen in the control group or their secondary outcomes and two to compare their results with other studies.

CONCLUSION

There is considerable between-study heterogeneity on how the Hawthorne effect relates to surgical outcomes. Further consideration on reporting and considering the importance of the Hawthorne effect in the design of surgical trials is warranted.

The quality of control groups in nonrandomized studies published in the Journal of Hand Surgery

PURPOSE

To evaluate control group selection in nonrandomized studies published in the Journal of Hand Surgery American (JHS).

METHODS

We reviewed all papers published in JHS in 2013 to identify studies that used nonrandomized control groups. Data collected included type of study design and control group characteristics. We then appraised studies to determine whether authors discussed confounding and selection bias and how they controlled for confounding.

RESULTS

Thirty-seven nonrandomized studies were published in JHS in 2013. The source of control was either the same institution as the study group, a different institution, a database, or not provided in the manuscript. Twenty-nine (78%) studies statistically compared key characteristics between control and study group. Confounding was controlled with matching, exclusion criteria, or regression analysis. Twenty-two (59%) papers explicitly discussed the threat of confounding and 18 (49%) identified sources of selection bias.

CONCLUSIONS

In our review of nonrandomized studies published in JHS, papers had well-defined controls that were similar to the study group, allowing for reasonable comparisons. However, we identified substantial confounding and bias that were not addressed as explicit limitations, which might lead the reader to overestimate the scientific validity of the data.

CLINICAL RELEVANCE

Incorporating a brief discussion of control group selection in scientific manuscripts should help readers interpret the study more appropriately. Authors, reviewers, and editors should strive to address this component of clinical importance.

Performance characteristics of the verbal QuickDASH

PURPOSE

To quantify the performance of the verbally administered Quick Disabilities of the Arm, Shoulder, and Hand (QuickDASH) questionnaire by assessing its replication of self-administered scores, its test-retest reliability, and its rate of scorable completion compared with its self-administered, written administration.

METHODS

Fifty patients presenting for initial visits to a hand clinic were enrolled regardless of diagnosis. All patients completed a written and a verbal QuickDASH 1 day apart (25 patients written first; 25 patients verbal first). Intraclass correlation coefficients quantified the verbal questionnaire's ability to reproduce written scores. Participants verbally completed the questionnaire a final time, 5 months later, to assess test-retest reliability. To quantify the usability of survey data, we compared percentages of scorable surveys between written and verbally administered QuickDASH questionnaires in this study and in prior studies within our division.

RESULTS

The intraclass correlation coefficient between the 2 QuickDASH administration types for the entire sample was 0.91. Across all participants, there was a minimal change in mean score from a patient's written QuickDASH to that patient's first verbal QuickDASH score. Scoring consistency between QuickDASH administrations was similar for each administration sequence (phone followed by written vs. written followed by phone) and by diagnosis. Test-retest reliability between the 2 verbal administrations demonstrated good reliability and a minimal difference between scores. In this study, no written or verbal surveys were incomplete. Reviewing our practice, 17% of 258 written questionnaires produced unscorable data compared with 0% of 239 verbally administered surveys.

CONCLUSIONS

Our results indicate that verbal administration of the QuickDASH replicates clinically relevant scores of the written QuickDASH, has good test-retest performance, and may minimize unusable data. These data allow researchers greater flexibility in gathering patient outcome data in both retrospective and prospective studies.

TYPE OF STUDY/LEVEL OF EVIDENCE

Diagnostic II.