Users' Guide to the Surgical Literature. Understanding confidence intervals

The Fragility of Statistically Significant Randomized Controlled Trials in Plastic Surgery

BACKGROUND

The fragility index has been proposed as a metric to evaluate the robustness of statistically significant findings in randomized controlled trials. It measures the number of events that a trial result relies on to maintain statistical significance. This study examines the robustness of statistically significant results from randomized controlled trials in the plastic surgery literature.

METHODS

A systematic literature search of the 15 highest impact plastic surgery journals was conducted to identify randomized controlled trials published between 2000 and 2017 that reported a statistically significant dichotomous outcome (p < 0.05). The fragility index of each study was calculated using Fisher's exact test. Multiple linear regression was used to determine trial characteristics associated with the fragility index.

RESULTS

The 90 eligible randomized controlled trials had a median sample size of 73.5 patients (25th to 75th percentile, 50 to 115) and a median of 20 events (25th to 75th percentile, 11 to 33.5) for the chosen outcome. The median fragility index was 1 (25th to 75th percentile, 0 to 4), indicating that statistical significance would be lost in half of the randomized controlled trials if a single patient had a change in event status. The fragility index was 0 in 24 of 90 (27 percent) randomized controlled trials, meaning the outcome immediately lost statistical significance on recalculation of the p value using Fisher's exact test.

CONCLUSIONS

The results of randomized controlled trials in plastic surgery demonstrate substantial fragility, as statistically significant results were found to hinge on a small number of events. The fragility index offers an intuitive and simple metric to complement the p value and determine the confidence in the results of randomized controlled trials.

A systematic review of questionnaires about patient's values and preferences in clinical practice guidelines

OBJECTIVE

We conducted a systematic review to evaluate questionnaires about patient's values and preferences to provide information on the most appropriate questionnaires to be used when developing clinical practice guidelines.

METHODS

A systematic literature search of the Cochrane Library, MEDLINE, Embase, Web of Science, Chinese Biomedical Database, China National Knowledge Infrastructure, and the Wanfang Database was performed to identify studies on questionnaires evaluating patient's values and preferences. The articles that used fully structured questionnaires or scales with standardized questions and answer options were included. We assessed the questionnaires' construction and content with a psychometric methodology and summarized the domains and items about patient's preferences and values.

RESULTS

A total of 7,008 records were retrieved by the search strategy and scanned, and 20 articles were finally included. Of these, 10 (50%) articles described the process of item generation and only four questionnaires (20%, 4/20) mentioned the pilot testing. Regarding "validity", seven questionnaires (35%, 7/20) assessed validity and only one (5%, 1/20) questionnaire assessed internal consistency, with Cornbrash's α values of 0.74-0.87. For "acceptability", the time to complete the questionnaires ranged from 10 to 30 minutes and only nine studies (45%, 9/20) reported the response rates. In addition, the results of domains and items about patient's preferences and values showed that the "effectiveness" domain was the most considered item in the patient's value questionnaire followed by "safety", "prognosis", and others, whereas the least considered domain was "physician's experience".

CONCLUSION

Only a few studies have developed questionnaires with rigorous psychometric methods to measure patient's preferences and values. Currently, still there is no valid or reliable questionnaire for patient's preferences and values for use when developing clinical practice guidelines. Further study should be conducted to develop standardized instruments to measure patient's preferences and values. This study provides the domains and items that may be used in formulating questionnaires about patient's preferences and values.

Users' guide to the surgical literature: how to assess a noninferiority trial

SUMMARY

A well-planned randomized controlled trial (RCT) is the most optimal study design to determine if a novel surgical intervention is any different than a prevailing one. Traditionally, when we want to show that a new surgical intervention is superior to a standard one, we analyze data from an RCT to see if the null hypothesis of "no difference" can be rejected (i.e., the 2 surgical interventions have the same effect). A noninferiority RCT design seeks to determine whether a new intervention is not worse than a prevailing (standard) one within an acceptable margin of risk or benefit, referred to as the "noninferiority margin." In the last decade, we have observed an increase in the publication of noninferiority RCTs. This article explores this type of study design and discusses the tools that can be used to appraise such a study.

Users' guide to the surgical literature: how to assess an article about harm in surgery

CLINICALSCENARIO

You are a new plastic surgeon in the community and you are referred a patient interested in breast reconstruction. The patient is a 35-year-old female school teacher who had a bilateral prophylactic mastectomy 2 years earlier, as she was a BRCA gene carrier. Since she is of a petite build with very little subcutaneous tissue or extra skin in the lower abdomen, you decide that she is not a suitable candidate for an abdomen-based autologous tissue reconstruction. You recommend the technique of tissue expansion and silicone gel implants. She is concerned, however, about the possibility of anaplastic large cell lymphoma (ALCL) developing in her breasts. She read in a magazine recently that ALCL, an unusual form of breast cancer, has been occurring in patients who have breast implants. She is very concerned that she might be at risk and asks for your opinion as to whether she should proceed with the procedure or not.

Measuring and understanding treatment effectiveness in hand surgery

Incorporating evidence-based medicine into practice is now an expectation for hand surgeons. Hand surgeons need to be able to assess associated benefits, risks, cost, and applicability of a treatment option when providing care to their patients. Using a clinical example, this article takes the reader through the three-step approach when using a publication from the medical literature on therapy. The focus of this article is primarily the second and third steps, which involve measuring and understanding treatment effectiveness.

Factors associated with post-intensive care unit adverse events: a clinical validation study

BACKGROUND

Many patients discharged from intensive care units (ICU) have complex care needs, placing them at risk of an adverse event in a ward environment. Currently, there is limited understanding of factors associated with these events in the post-intensive care population. A recent study explored intensive care liaison nurses' opinions on factors associated with these events; 25 factors were identified, highlighting the multifaceted nature of post-intensive care adverse events.

AIM

This study aimed to clinically validate 25 factors intensive care liaison nurses believe are associated with post-intensive care adverse events, to determine the factors' relevance and importance to clinical practice.

DESIGN

Prospective, clinical validation study.

METHOD

Data were prospectively collected on a convenience sample of 52 patients at 4 tertiary referral hospitals in an Australian capital city. All patients had experienced an adverse event after intensive care discharge.

RESULTS

Each of the 25 factors contributed to adverse events in at least 6 patients. The factors associated with the most adverse events were those that related to the patient such as illness severity and co-morbidities.

CONCLUSION

Clinical care and research should focus on modifiable factors in care processes to reduce the risk of future adverse events in post-intensive care patients.

RELEVANCE TO CLINICAL PRACTICE

Many patients are at risk of post-ICU adverse events due to the contribution of non-modifiable factors. However, by focusing on modifiable factors in care processes, the risk of post-ICU adverse events may be reduced.

Is low-level laser therapy in relieving neck pain effective? Systematic review and meta-analysis

The aim of this study is to determine the efficacy of low-level laser therapy (LLLT) in reducing acute and chronic neck pain as measured by the visual analog scale (VAS). A systematic search of nine electronic databases was conducted to identify original articles. For study selection, two reviewers independently assessed titles, abstracts, and full text for eligibility. Methodological quality was assessed using the Detsky scale. Data were analyzed using random-effects model in the presence of heterogeneity and fixed-effect model in its absence. Heterogeneity was assessed using Cochran's Q statistic and quantifying I (2). Risk ratios (RR) with 95 % confidence intervals (CI) were reported. Eight randomized controlled trials involving 443 patients met the strict inclusion criteria. Inter-rater reliability for study selection was 92.8 % (95 % CIs 80.9-100 %) and for methodological quality assessment was 83.9 % (95 % CIs 19.4-96.8 %). Five trials included patients with cervical myofascial pain syndrome (CMPS), and three trials included different patient populations. A meta-analysis of five CMPS trials revealed a mean improvement of VAS score of 10.54 with LLLT (95 % CI 0.37-20.71; Heterogeneity I (2 )= 65 %, P = 0.02). This systematic review provides inconclusive evidence because of significant between-study heterogeneity and potential risk of bias. The benefit seen in the use of LLLT, although statistically significant, does not constitute the threshold of minimally important clinical difference.