Why proper understanding of confidence intervals and statistical significance is important

Transforming nursing assessment in acute hospitals: A cluster randomised controlled trial of an evidence-based nursing core assessment (the ENCORE trial)

BACKGROUND

Patient safety is threatened when early signs of clinical deterioration are missed or not acted upon. This research began as a clinical-academic partnership established around a shared concern of nursing physical assessment practices on general wards and delayed recognition of clinical deterioration. The outcome was the development of a complex intervention facilitated at the ward level for proactive nursing surveillance.

METHODS

The evidence-based nursing core assessment (ENCORE) trial was a pragmatic cluster-randomised controlled trial. We hypothesised that ward intervention would reduce the incidence of patient rescue events (medical emergency team activations) and serious adverse events. We randomised 29 general wards in a 1:2 allocation, across 5 Australian hospitals to intervention (n = 10) and usual care wards (n = 19). Skilled facilitation over 12 months enabled practitioner-led, ward-level practice change for proactive nursing surveillance. The primary outcome was the rate of medical emergency team activations and secondary outcomes were unplanned intensive care unit admissions, on-ward resuscitations, and unexpected deaths. Outcomes were prospectively collected for 6 months following the initial 6 months of implementation. Analysis was at the patient level using generalised linear mixed models to account for clustering by ward.

RESULTS

We analysed 29,385 patient admissions to intervention (n = 11,792) and control (n = 17,593) wards. Adjusted models for overall effects suggested the intervention increased the rate of medical emergency team activations (adjusted incidence rate ratio 1.314; 95 % confidence interval 0.975, 1.773), although the confidence interval was compatible with a marginal decrease to a substantial increase in rate. Confidence intervals for secondary outcomes included a range of plausible effects from benefit to harm. However, considerable heterogeneity was observed in intervention effects by patient comorbidity. Among patients with few comorbid conditions in the intervention arm there was a lower medical emergency team activation rate and decreased odds of unexpected death. Among patients with multimorbidity in the intervention arm there were higher rates of medical emergency team activation and intensive care unit admissions.

CONCLUSION

Trial outcomes have refined our assumptions about the impact of the ENCORE intervention. The intervention appears to have protective effects for patients with low complexity where frontline teams can respond locally. It also appears to have redistributed medical emergency team activations and unplanned intensive care unit admissions, mobilising higher rates of rescue for patients with multimorbidity.

TRIAL REGISTRATION NUMBER

ACTRN12618001903279 (Date of registration: 22/11/2018; First participant recruited: 01/02/2019).

Involving patients and the public In sTatistIcal Analysis pLans (INITIAL): A delphi survey

BACKGROUND

Patient and public involvement (PPI) in trials aims to enhance research by improving its relevance and transparency. Planning for statistical analysis begins at the design stage of a trial within the protocol and is refined and detailed in a Statistical Analysis Plan (SAP). While PPI is common in design and protocol development it is less common within SAPs. This study aimed to reach consensus on the most important and relevant statistical analysis items within an SAP to involve patients and the public.

METHODS

We developed a UK-based, two-round Delphi survey through an iterative consultation with public partners, statisticians, and trialists. The consultation process started with 55 items from international guidance for statistical analysis plans. We aimed to recruit at least 20 participants per key stakeholder group for inclusion in the final analysis of the Delphi survey. Participants were asked to vote on each item using a Likert scale from 1 to 9, where a rating of 1 to 3 was labelled as having 'limited importance'; 4 to 6 as 'important but not critical' and 7 to 9 as 'critical' to involve patients and the public. Results from the second round determined consensus on critical items for PPI.

RESULTS

The consultation exercise led to the inclusion of 15 statistical items in the Delphi survey. We recruited 179 participants, of whom 72% (129: 36 statisticians, 29 patients or public partners, 25 clinical researchers or methodologists, 27 trial managers, and 12 PPI coordinators) completed both rounds. Participants were on average 48 years old, 60% were female, 84% were White, 64% were based in England and 84% had at least five years' experience in trials. Four items reached consensus regarding critical importance for patient and public involvement: presentation of results to trial participants; summary and presentation of harms; interpretation and presentation of findings in an academic setting; factors impacting how well a treatment works. No consensus was reached for the remaining 11 items. In general, the results were consistent across stakeholder groups.

DISCUSSION

We identified four critical items to involve patients and the public in statistical analysis plans. The remaining 11 items did not reach consensus and need to be considered in a case-by-case basis with most responders considering patient and public involvement important (but not critical). Our research provides a platform to enable focused future efforts to improve patient and public involvement in trials and enhance the relevance of statistical analyses to patients and the public.

A re-analysis of 150 women's health trials to investigate how the Bayesian approach may offer a solution to the misinterpretation of statistical findings

OBJECTIVE

To investigate whether a Bayesian interpretation might help prevent misinterpretation of statistical findings and support authors to differentiate evidence of no effect from statistical uncertainty.

DESIGN

A Bayesian re-analysis to determine posterior probabilities of clinically important effects (e.g., a large effect is set at a 4 percentage point difference and a trivial effect to be within a 0.5 percentage point difference). Posterior probabilities greater than 95% are considered as strong statistical evidence, and less than 95% as inconclusive.

SAMPLE

150 major women's health trials with binary outcomes.

MAIN OUTCOME MEASURES

Posterior probabilities of large, moderate, small and trivial effects.

RESULTS

Under frequentist methods, 48 (32%) were statistically significant (p-value ≤ 0.05) and 102 (68%) statistically non-significant. The frequentist and Bayesian point estimates and confidence intervals showed strong concordance. Of the statistically non-significant trials (n = 102), the Bayesian approach classified the majority (94, 92%) as inconclusive, neither able to confirm or refute effectiveness. A small number of statistically non-significant findings (8, 8%) were classified as having strong statistical evidence of an effect.

CONCLUSIONS

Whilst almost all trials report confidence intervals, in practice most statistical findings are interpreted on the basis of statistical significance, mostly concluding evidence of no effect. Findings here suggest the majority are likely uncertain. A Bayesian approach could help differentiate evidence of no effect from statistical uncertainty.

Replacing of sedentary behavior with physical activity and the risk of mortality in people with prediabetes and diabetes: a prospective cohort study

BACKGROUND

Sedentary behavior is prevalent among people with diabetes and is associated with unfavorable cardiometabolic health. However, there is limited evidence regarding the impact of replacing sedentary time (ST) with physical activity on mortality in people with prediabetes and diabetes. We prospectively examined the association between accelerometer-measured ST and mortality among people with prediabetes and diabetes after adjusting for demographic characteristics, lifestyle factors, and moderate- to vigorous-intensity PA (MVPA). We further determined the effect of replacing ST with equal time of different types of physical activities on all-cause mortality.

METHODS

We included 1242 adults with prediabetes and 1037 with diabetes from the National Health and Nutrition Examination Survey. Restricted cubic splines were fitted to determine the dose-response association between ST and overall mortality. Isotemporal substitution modeling was used to explore the hazard ratio (HR) effects of ST replacement.

RESULTS

During a median follow-up of 14.1 years, 424 adults with prediabetes and 493 with diabetes died. Compared with the lowest tertile of ST, the multivariable-adjusted HRs for all-cause mortality in the highest tertile were 1.76 (95% confidence interval [CI] 1.19, 2.60) for participants with prediabetes and 1.76 (1.17, 2.65) for those with diabetes. Additionally, a linear association between ST and all-cause mortality was observed in adults with prediabetes and diabetes, with HRs for each 60 min/day increment in ST of 1.19 (1.10, 1.30) and 1.25 (1.12, 1.40), respectively. Isotemporal substitution results indicated that individuals with prediabetes whose ST was replaced by 30 min of light-intensity physical activity (LPA) and MVPA had 9% and 40% lower all-cause mortality, respectively. In people with diabetes, replacing sedentary behavior with an equivalent time of LPA and MVPA was also associated with mortality risk reduction (HR 0.89; 95% CI 0.84, 0.95 for LPA; HR 0.73; 95% CI 0.49, 1.11 for MVPA).

CONCLUSIONS

Higher ST was associated in a dose-response manner with an increased risk of premature mortality among adults with prediabetes and diabetes. Statistically replacing ST with LPA was potentially beneficial for health in this high-risk population.

Swimming sprint performance depends on upper/lower limbs strength and swimmers level

Swimming performance is likely influenced by strength, but differences between butterfly, backstroke, breaststroke and front crawl, as well as between novice and expert swimmers, are unclear. We have examined the associations between sprint performances, upper and lower limb strength, and anthropometric characteristics in 14 (six males and eight females) non-elite and 16 (nine males and seven females) elite-level swimmers. After an anthropometric characterisation, participants performed four 25 m maximal swims (one per technique) with 10 min intervals, right and left shoulder flexion/extension isokinetic testing at 90 and 300º/s angular velocities and three countermovement jumps. Pearson correlation analysis showed that sprint times were moderate-largely negatively correlated with upper and lower limb strength and power (r ± 95%CI = 0.39 ± 0.26-0.77 ± 0.13, p < 0.05). Elite swimmers higher strength levels were associated with longer stroke length in butterfly and front crawl, and with higher stroke rate in backstroke and breaststroke (r ± 95%CI = 0.37 ± 0.32-0.68 ± 0.21; p < 0.05). Butterfly, backstroke and front crawl sprint times were moderate-largely negatively related with arm span (r ± 95%CI = 0.37 ± 0.26, 0.39 ± 0.25 and 0.69 ± 0.17, p < 0.05). The predictive model indicated that higher dry-land strength values distinguished elite from non-elite swimmers (r2 = 0.67-0.81; p < 0.001). This association was not observed per performance level and per sex, confirming that sprint swimming performance levels can be differentiated by dry-land strength testing.

The effectiveness of a mobile application-based programme for rehabilitation after total hip or knee arthroplasty: A randomised controlled trial

BACKGROUND

Total hip arthroplasty and total knee arthroplasty are widely performed worldwide. Patients undergoing total hip or knee arthroplasty are often discharged after a short hospital stay. Using information and communication technologies, such as mobile applications, to provide rehabilitation services remotely may be a strategy to support patients' postoperative recovery.

OBJECTIVE

This study aimed to evaluate the effectiveness of a rehabilitation programme delivered via a mobile application among Chinese patients after total hip or knee arthroplasty.

DESIGN

Randomised controlled trial.

SETTING

A teaching hospital in Shanghai, China.

PARTICIPANTS

Eighty-six patients who received a unilateral primary total hip or knee arthroplasty.

METHODS

The participants were recruited in the hospital and randomised into either the experimental or control group. Once discharged from the hospital, the control group (n = 43) received usual care, and the experimental group (n = 43) received usual care plus a 6-week mobile rehabilitation programme. Outcomes were assessed three times: the day before hospital discharge and 6 and 10 weeks after discharge. Primary outcomes were the changes in scores of self-efficacy and patient-reported physical function from baseline to 6 weeks post-discharge. Secondary outcomes included changes in scores of pain, depression, anxiety, and health-related quality of life. Data were analysed using generalised estimating equations.

RESULTS

At 6 weeks after hospital discharge, the experimental group showed statistically significant improvements compared to the control in scores of self-efficacy (adjusted mean difference = 0.72, 95% CI 0.31 to 1.14, P < .001) and patient-reported physical function (adjusted mean difference = 4.57, 95% CI 1.24 to 7.90, P = .007). The between-group difference in self-efficacy probably reached clinical significance. At week-10 follow-up, the experimental group had statistically significant improvements in scores of self-efficacy (adjusted mean difference = 0.64, 95% CI 0.33 to 0.95, P < .001), health-related quality of life (adjusted mean difference = 0.06, 95% CI 0.01 to 0.10, P = .018), anxiety (adjusted mean difference = -0.51, 95% CI -0.91 to -0.10, P = .015), and depression (adjusted mean difference = -0.37, 95% CI -0.66 to -0.08, P = .012). The between-group difference in self-efficacy and health-related quality of life may be clinically significant.

CONCLUSION

Mobile application-based rehabilitation demonstrated potentially positive effects on patients' self-efficacy, patient-reported physical function, health-related quality of life, and levels of anxiety and depression.

TRIAL REGISTRATION

Registered with the Australian New Zealand Clinical Trials Registry on 6 July 2021 (ACTRN12621000867897).