Determination of the clinical importance of study results

Six-minute-walk test in idiopathic pulmonary fibrosis: test validation and minimal clinically important difference

RATIONALE

The 6-minute-walk test (6MWT) is a practical and clinically meaningful measure of exercise tolerance with favorable performance characteristics in various cardiac and pulmonary diseases. Performance characteristics in patients with idiopathic pulmonary fibrosis (IPF) have not been systematically evaluated.

OBJECTIVES

To assess the reliability, validity, and responsiveness of the 6MWT and estimate the minimal clinically important difference (MCID) in patients with IPF.

METHODS

The study population included all subjects completing a 6MWT in a clinical trial evaluating interferon gamma-1b (n = 822). Six-minute walk distance (6MWD) and other parameters were measured at baseline and at 24-week intervals using a standardized protocol. Parametric and distribution-independent correlation coefficients were used to assess the strength of the relationships between 6MWD and measures of pulmonary function, dyspnea, and health-related quality of life. Both distribution-based and anchor-based methods were used to estimate the MCID.

MEASUREMENTS AND MAIN RESULTS

Comparison of two proximal measures of 6MWD (mean interval, 24 d) demonstrated good reliability (coefficient = 0.83; P < 0.001). 6MWD was weakly correlated with measures of physiologic function and health-related quality of life; however, values were consistently and significantly lower for patients with the poorest functional status, suggesting good construct validity. Importantly, change in 6MWD was highly predictive of mortality; a 24-week decline of greater than 50 m was associated with a fourfold increase in risk of death at 1 year (hazard ratio, 4.27; 95% confidence interval, 2.57- 7.10; P < 0.001). The estimated MCID was 24-45 m.

CONCLUSIONS

The 6MWT is a reliable, valid, and responsive measure of disease status and a valid endpoint for clinical trials in IPF.

What differences are detected by superiority trials or ruled out by noninferiority trials? A cross-sectional study on a random sample of two-hundred two-arms parallel group randomized clinical trials

BACKGROUND

The smallest difference to be detected in superiority trials or the largest difference to be ruled out in noninferiority trials is a key determinant of sample size, but little guidance exists to help researchers in their choice. The objectives were to examine the distribution of differences that researchers aim to detect in clinical trials and to verify that those differences are smaller in noninferiority compared to superiority trials.

METHODS

Cross-sectional study based on a random sample of two hundred two-arm, parallel group superiority (100) and noninferiority (100) randomized clinical trials published between 2004 and 2009 in 27 leading medical journals. The main outcome measure was the smallest difference in favor of the new treatment to be detected (superiority trials) or largest unfavorable difference to be ruled out (noninferiority trials) used for sample size computation, expressed as standardized difference in proportions, or standardized difference in means. Student t test and analysis of variance were used.

RESULTS

The differences to be detected or ruled out varied considerably from one study to the next; e.g., for superiority trials, the standardized difference in means ranged from 0.007 to 0.87, and the standardized difference in proportions from 0.04 to 1.56. On average, superiority trials were designed to detect larger differences than noninferiority trials (standardized difference in proportions: mean 0.37 versus 0.27, P = 0.001; standardized difference in means: 0.56 versus 0.40, P = 0.006). Standardized differences were lower for mortality than for other outcomes, and lower in cardiovascular trials than in other research areas.

CONCLUSIONS

Superiority trials are designed to detect larger differences than noninferiority trials are designed to rule out. The variability between studies is considerable and is partly explained by the type of outcome and the medical context. A more explicit and rational approach to choosing the difference to be detected or to be ruled out in clinical trials may be desirable.

[Recommendations for the clinical evaluation of results in the biomedical literature]

The assessment and interpretation of the results of a clinical study are a real challenge for the clinicians. In this paper we establish a general basis for a critical and reserved assessment of these, from the fundamental aspects of the design and statistics, as well as the application of the results to our own patients according to risk and benefit criteria. Main errors and the traps that should be avoided are emphasised.

Preoperative fasting for preventing perioperative complications in children

BACKGROUND

Children, like adults, are required to fast before general anaesthesia with the aim of reducing the volume and acidity of their stomach contents. It is thought that fasting reduces the risk of regurgitation and aspiration of gastric contents during surgery. Recent developments have encouraged a shift from the standard 'nil-by-mouth-from-midnight' fasting policy to more relaxed regimens. Practice has been slow to change due to questions relating to the duration of a total fast, the type and amount of intake permitted.

OBJECTIVES

To systematically assess the effects of different fasting regimens (duration, type and volume of permitted intake) and the impact on perioperative complications and patient well being (aspiration, regurgitation, related morbidity, thirst, hunger, pain, comfort, behaviour, nausea and vomiting) in children.

SEARCH STRATEGY

We searched Cochrane Wounds Group Specialised Register (searched 25/6/09), the Cochrane Central Register of Controlled Trials (The Cochrane Library, Issue 2 2009), Ovid MEDLINE (1950 to June Week 2 2009), Ovid EMBASE (1980 to 2009 Week 25), EBSCO CINAHL (1982 to June Week 3 2009), the National Research Register, relevant conference proceedings and article reference lists and contacted experts.

SELECTION CRITERIA

Randomised and quasi randomised controlled trials of preoperative fasting regimens for children were identified.

DATA COLLECTION AND ANALYSIS

Data extraction and trial quality assessment was conducted independently by three authors. Trial authors were contacted for additional information including adverse events.

MAIN RESULTS

This first update of the review identified two additional eligible studies, bringing the total number of included studies to 25 (forty seven randomised controlled comparisons involving 2543 children considered to be at normal risk of regurgitation or aspiration during anaesthesia). Only one incidence of aspiration and regurgitation was reported.Children permitted fluids up to 120 minutes preoperatively were not found to experience higher gastric volumes or lower gastric pH values than those who fasted. The children permitted fluids were less thirsty and hungry, better behaved and more comfortable than those who fasted.Clear fluids preoperatively did not result in a clinically important difference in children's gastric volume or pH. Evidence relating to the preoperative intake of milk was sparse. The volume of fluid permitted during the preoperative period did not appear to impact on children's intraoperative gastric volume or pH contents.

AUTHORS' CONCLUSIONS

There is no evidence that children who are denied oral fluids for more than six hours preoperatively benefit in terms of intraoperative gastric volume and pH compared with children permitted unlimited fluids up to two hours preoperatively. Children permitted fluids have a more comfortable preoperative experience in terms of thirst and hunger. This evidence applies only to children who are considered to be at normal risk of aspiration/regurgitation during anaesthesia.

People with low back pain typically need to feel 'much better' to consider intervention worthwhile: an observational study

QUESTIONS

How much of an effect do five common physiotherapy interventions need to have for patients with low back pain to perceive they are worth their cost, discomfort, risk, and incovenience? Are there any differences between the interventions? Do specific characteristics of people with low back pain predict the smallest important difference?

DESIGN

Cross-sectional, observational study.

PARTICIPANTS

77 patients with non-specific low back pain who had not yet commenced physiotherapy intervention.

OUTCOME MEASURES

The smallest worthwhile effect was measured in terms of global perceived change (0 to 4) and percentage perceived change.

RESULTS

Participants perceived that intervention would have to make them 'much better', which corresponded to 1.7 (SD 0.7) on the 4-point scale, or improve their symptoms by 42% (SD 23), to make it worthwhile. There was little distinction made between interventions, regardless of whether smallest worthwhile effects were quantified as global perceived change (p = 0.09) or percentage perceived change (p = 1.00). Severity of symptoms independently (p = 0.01) predicted percentage perceived change explaining 9% of the variance, so that for each increase in severity of symptoms of 1 point out of 10 there was an increase of 4% in the percentage perceived change that participants considered would make intervention worthwhile.

CONCLUSIONS

Typically people with low back pain feel that physiotherapy intervention must reduce their symptoms by 42%, or make them feel 'much better' for intervention to be worthwhile.

How Can We Assess Outcomes of Clinical Trials: The MCID Approach

Interpreting changes in outcomes of clinical trials in chronic obstructive pulmonary disease should be viewed from a broader perspective than only the statistical significance of the findings. The minimal clinical difference in outcome measures provides a conceptual framework to assist in clinical trial interpretation and a methodology to assess the clinical relevance of study results. Use of distribution-based techniques, comparison with other external measures, and opinions from experts, clinicians and patients can assist in minimal clinically important difference development. Although the minimal clinically important difference has been suggested for a wide range of outcomes of importance in chronic obstructive pulmonary disease, many have not been subjected to rigorous analysis. For newer tools such as activity monitors and questionnaires and measures not widely employed such as laboratory-based exercise tests, minimal clinically important differences remain to be determined.

Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials

OBJECTIVE

Randomized trials generally use "frequentist" statistics based on P-values and 95% confidence intervals. Frequentist methods have limitations that might be overcome, in part, by Bayesian inference. To illustrate these advantages, we re-analyzed randomized trials published in four general medical journals during 2004.

STUDY DESIGN AND SETTING

We used Medline to identify randomized superiority trials with two parallel arms, individual-level randomization and dichotomous or time-to-event primary outcomes. Studies with P<0.05 in favor of the intervention were deemed "positive"; otherwise, they were "negative." We used several prior distributions and exact conjugate analyses to calculate Bayesian posterior probabilities for clinically relevant effects.

RESULTS

Of 88 included studies, 39 were positive using a frequentist analysis. Although the Bayesian posterior probabilities of any benefit (relative risk or hazard ratio<1) were high in positive studies, these probabilities were lower and variable for larger benefits. The positive studies had only moderate probabilities for exceeding the effects that were assumed for calculating the sample size. By comparison, there were moderate probabilities of any benefit in negative studies.

CONCLUSION

Bayesian and frequentist analyses complement each other when interpreting the results of randomized trials. Future reports of randomized trials should include both.

Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations

A consensus meeting was convened by the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) to provide recommendations for interpreting clinical importance of treatment outcomes in clinical trials of the efficacy and effectiveness of chronic pain treatments. A group of 40 participants from universities, governmental agencies, a patient self-help organization, and the pharmaceutical industry considered methodologic issues and research results relevant to determining the clinical importance of changes in the specific outcome measures previously recommended by IMMPACT for 4 core chronic pain outcome domains: (1) Pain intensity, assessed by a 0 to 10 numerical rating scale; (2) physical functioning, assessed by the Multidimensional Pain Inventory and Brief Pain Inventory interference scales; (3) emotional functioning, assessed by the Beck Depression Inventory and Profile of Mood States; and (4) participant ratings of overall improvement, assessed by the Patient Global Impression of Change scale. It is recommended that 2 or more different methods be used to evaluate the clinical importance of improvement or worsening for chronic pain clinical trial outcome measures. Provisional benchmarks for identifying clinically important changes in specific outcome measures that can be used for outcome studies of treatments for chronic pain are proposed.

PERSPECTIVE

Systematically collecting and reporting the recommended information needed to evaluate the clinical importance of treatment outcomes of chronic pain clinical trials will allow additional validation of proposed benchmarks and provide more meaningful comparisons of chronic pain treatments.

Preoperative fasting for preventing perioperative complications in children

BACKGROUND

Children, like adults, are required to fast before general anaesthesia with the aim of reducing the volume and acidity of their stomach contents. It is thought that fasting reduces the risk of regurgitation and aspiration of gastric contents during surgery. Recent developments have encouraged a shift from the standard 'nil-by-mouth-from-midnight' fasting policy to more relaxed regimens. Practice has been slow to change due to questions relating to the duration of a total fast, the type and amount of intake permitted.

OBJECTIVES

To systematically assess the effects of different fasting regimens (duration, type and volume of permitted intake) and the impact on perioperative complications and patient wellbeing (aspiration, regurgitation, related morbidity, thirst, hunger, pain, comfort, behaviour, nausea and vomiting) in children.

SEARCH STRATEGY

We searched Cochrane Wounds Group Specialised Register, the Cochrane Central Register of Controlled Trials, MEDLINE, CINAHL, the National Research Register, relevant conference proceedings and article reference lists and contacted experts.

SELECTION CRITERIA

Randomised and quasi randomised controlled trials of preoperative fasting regimens for children were identified.

DATA COLLECTION AND ANALYSIS

Data extraction and trial quality assessment was conducted independently by two authors. Trial authors were contacted for additional information including adverse events.

MAIN RESULTS

Forty-three randomised controlled comparisons (from 23 trials) involving 2350 children considered to be at normal risk of regurgitation or aspiration during anaesthesia. Only one incidence of aspiration and regurgitation was reported. Children permitted fluids up to 120 minutes preoperatively were not found to experience higher gastric volumes or lower gastric pH values than those who fasted. The children permitted fluids were also less thirsty and hungry, better behaved and more comfortable than those who fasted. Clear fluids preoperatively did not result in a clinically important difference in the children's gastric volume or pH. Evidence relating to the preoperative intake of milk was sparse. The volume of fluid permitted during the preoperative period did not appear to impact on children's intraoperative gastric volume or pH contents.

AUTHORS' CONCLUSIONS

There is no evidence that children who are not permitted oral fluids for more than six hours preoperatively benefit in terms of intraoperative gastric volume and pH over children permitted unlimited fluids up to two hours preoperatively. Children permitted fluids have a more comfortable preoperative experience in terms of thirst and hunger. This evidence applies only to children who are considered to be at normal risk of aspiration/regurgitation during anaesthesia.

Reporting point and interval estimates of effect-size for planned contrasts: fixed within effect analyses of variance

The purpose of this tutorial is threefold: (a) review the state of statistical science regarding effect-sizes, (b) illustrate the importance of effect-sizes for interpreting findings in all forms of research and particularly for results of clinical-outcome research, and (c) demonstrate just how easily a criterion on reporting effect-sizes in research manuscripts can be accomplished. The presentation centers on within-effect analyses of variance including the one-way design for testing pre-post hypotheses and the two-way parallel-groups design for making direct comparisons of competing treatment protocols (e.g., experimental treatment versus control). The presentation is supported with worked examples and a web site containing templates for software applications.

EDUCATIONAL OBJECTIVES

The reader will be able to: (1) explain the rationale for the increased use of estimates of effect-size in reporting results in published research manuscripts; (2) describe what an effect-size is (generally considered) and provide a rationale for its importance; (3) distinguish among the many forms of effect-size and apply their features to the most appropriate choices under specific research circumstances; and (4) appropriately report and interpret effect-sizes.

The 13- and 20-item Hopkins Symptom Checklist Depression Scale: psychometric properties in primary care patients with minor depression or dysthymia

OBJECTIVE

Depression scales that are responsive to changes in clinical symptoms are important for clinical monitoring and outcomes assessment in longitudinal studies. We evaluated the psychometric properties and responsiveness to clinical change of the 13- and 20-item versions of the Hopkins Symptom Checklist Depression Scale (HSCL-D).

METHODS

A secondary data analysis from a large 11-week, multicenter clinical trial, comparing three treatments was performed. Adult patients with minor depression or dysthymia and a score of > or = 10 on the Hamilton Depression Rating Scale (HDRS) were recruited from primary care clinics. Item-total correlations and Cronbach alphas were computed for HSCL-D-13 and HSCL-D-20. Clinical response at 11 weeks was defined by a Hamilton Depression Rating Scale (HDRS) < 10, clinical remission by a HDRS < 7, and criterion symptom remission by < or = 1 DSM-III-R criterion symptoms. Standardized effect sizes and Guyatt's responsiveness statistic were determined for the 13- and 20-item HSCL-D.

RESULTS

Of the 656 subjects enrolled, 511 (77.9%) had complete data and were included in the analysis. Patients were 61.1 +/- 15.0 years old; minor depression was diagnosed in 238, dysthymia in 273. Both scales had good internal consistency; Cronbach's alpha = 0.835 and 0.859 for the 13- and 20-items questionnaires respectively. Standardized effect sizes for clinical response (0.62 for the HSCL-D-13; 0.66 for the HSCL-D-20), clinical remission (0.69 and 0.70), and criterion symptom remission (0.65 and 0.67) showed moderate to large effects and did not differ significantly for the two versions. Responsiveness was virtually identical for patients with minor depression and dysthymia but responsiveness was substantially lower for ethnic minorities.

CONCLUSION

The HSCL-D-13 and 20-item versions have similar responsiveness to change. For use in European Americans, we recommend the HSCL-D-13 if response burden is the preeminent consideration. To more fully capture DSM criterion symptoms, we recommend the HSCL-D-20.

A pilot clinical trial of oral pentosan polysulfate and oral hydroxyzine in patients with interstitial cystitis

PURPOSE

This pilot study was designed to evaluate the feasibility of a multicenter, randomized, clinical trial in interstitial cystitis (IC). Secondary objectives were to evaluate the safety and efficacy of oral pentosan polysulfate sodium (PPS), hydroxyzine, and the combination to consider their use in a larger randomized clinical trial.

MATERIALS AND METHODS

A 2 x 2 factorial study design was used to evaluate PPS and hydroxyzine. Participants met the National Institutes of Health-National Institute for Diabetes and Digestive and Kidney Diseases criteria for IC and reported at least moderate pain and frequency for a minimum of 6 months before study entry. The primary end point was a patient reported global response assessment. Secondary end points included validated symptom indexes and patient reports of pain, urgency and frequency. The target sample size was 136 participants recruited during 10 months.

RESULTS

A total of 121 (89% of goal) participants were randomized over 18 months and 79% provided complete followup data. The response rate for hydroxyzine was 31% for those treated and 20% for those not treated (p = 0.26). A nonsignificant trend was seen in the PPS treatment groups (34%) as compared to no PPS (18%, p = 0.064). There were no treatment differences for any of the secondary end points. Adverse events were mostly minor and similar to those in previous reports.

CONCLUSIONS

The low global response rates for PPS and hydroxyzine suggest that neither provided benefit for the majority of patients with IC. This trial demonstrated the feasibility of conducting a multicenter randomized clinical trial in IC using uniform procedures and outcomes. However, slow recruitment underscored the difficulties of evaluating commonly available IC drugs.