Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines

Prognostic models play a crucial role in the clinical decision-making process. Unfortunately, missing covariate data impede the construction of valid and reliable models, potentially introducing bias, if handled inappropriately. The extent of missing covariate data within reported cancer prognostic studies, the current handling and the quality of reporting this missing covariate data are unknown. Therefore, a review was conducted of 100 articles reporting multivariate survival analyses to assess potential prognostic factors, published within seven cancer journals in 2002. Missing covariate data is a common occurrence in studies performing multivariate survival analyses, being apparent in 81 of the 100 articles reviewed. The percentage of eligible cases with complete data was obtainable in 39 articles, and was <90% in 17 of these articles. The methods used to handle incomplete covariates were obtainable in 32 of the 81 articles with known missing data and the most commonly reported approaches were complete case and available case analysis. This review has highlighted deficiencies in the reporting of missing covariate data. Guidelines for presenting prognostic studies with missing covariate data are proposed, which if followed should clarify and standardise the reporting in future articles.

Evaluating non-randomised intervention studies

OBJECTIVES

To consider methods and related evidence for evaluating bias in non-randomised intervention studies.

DATA SOURCES

Systematic reviews and methodological papers were identified from a search of electronic databases; handsearches of key medical journals and contact with experts working in the field. New empirical studies were conducted using data from two large randomised clinical trials.

METHODS

Three systematic reviews and new empirical investigations were conducted. The reviews considered, in regard to non-randomised studies, (1) the existing evidence of bias, (2) the content of quality assessment tools, (3) the ways that study quality has been assessed and addressed. (4) The empirical investigations were conducted generating non-randomised studies from two large, multicentre randomised controlled trials (RCTs) and selectively resampling trial participants according to allocated treatment, centre and period.

RESULTS

In the systematic reviews, eight studies compared results of randomised and non-randomised studies across multiple interventions using meta-epidemiological techniques. A total of 194 tools were identified that could be or had been used to assess non-randomised studies. Sixty tools covered at least five of six pre-specified internal validity domains. Fourteen tools covered three of four core items of particular importance for non-randomised studies. Six tools were thought suitable for use in systematic reviews. Of 511 systematic reviews that included non-randomised studies, only 169 (33%) assessed study quality. Sixty-nine reviews investigated the impact of quality on study results in a quantitative manner. The new empirical studies estimated the bias associated with non-random allocation and found that the bias could lead to consistent over- or underestimations of treatment effects, also the bias increased variation in results for both historical and concurrent controls, owing to haphazard differences in case-mix between groups. The biases were large enough to lead studies falsely to conclude significant findings of benefit or harm. Four strategies for case-mix adjustment were evaluated: none adequately adjusted for bias in historically and concurrently controlled studies. Logistic regression on average increased bias. Propensity score methods performed better, but were not satisfactory in most situations. Detailed investigation revealed that adequate adjustment can only be achieved in the unrealistic situation when selection depends on a single factor.

CONCLUSIONS

Results of non-randomised studies sometimes, but not always, differ from results of randomised studies of the same intervention. Non-randomised studies may still give seriously misleading results when treated and control groups appear similar in key prognostic factors. Standard methods of case-mix adjustment do not guarantee removal of bias. Residual confounding may be high even when good prognostic data are available, and in some situations adjusted results may appear more biased than unadjusted results. Although many quality assessment tools exist and have been used for appraising non-randomised studies, most omit key quality domains. Healthcare policies based upon non-randomised studies or systematic reviews of non-randomised studies may need re-evaluation if the uncertainty in the true evidence base was not fully appreciated when policies were made. The inability of case-mix adjustment methods to compensate for selection bias and our inability to identify non-randomised studies that are free of selection bias indicate that non-randomised studies should only be undertaken when RCTs are infeasible or unethical. Recommendations for further research include: applying the resampling methodology in other clinical areas to ascertain whether the biases described are typical; developing or refining existing quality assessment tools for non-randomised studies; investigating how quality assessments of non-randomised studies can be incorporated into reviews and the implications of individual quality features for interpretation of a review's results; examination of the reasons for the apparent failure of case-mix adjustment methods; and further evaluation of the role of the propensity score.

Applying the right statistics: analyses of measurement studies

The study of measurement error, observer variation and agreement between different methods of measurement are frequent topics in the imaging literature. We describe the problems of some applications of correlation and regression methods to these studies, using recent examples from this literature. We use a simulated example to show how these problems and misinterpretations arise. We describe the 95% limits of agreement approach and a similar, appropriate, regression technique. We discuss the difference vs. mean plot, and the pitfalls of plotting difference against one variable only. We stress that these are questions of estimation, not significance tests, and show how confidence intervals can be found for these estimates.

Effect of arterial revascularisation on survival: a systematic review of studies comparing bilateral and single internal mammary arteries

BACKGROUND

Coronary artery bypass grafting (CABG) is the commonest major operation in most developed countries. A single internal mammary artery (IMA) graft has proven survival benefits, but the additional survival advantage of a second graft is unknown. We systematically reviewed published studies of bilateral versus single IMA grafts in CABG to assess any differences in survival.

METHODS

We identified from Medline all studies in which single and bilateral IMA grafts were compared. We included studies in which at least 100 patients in each group had been followed up for at least 4 years. We assessed study quality on the basis of patient selection, comparability of intervention groups (especially for age, sex, ventricular function, and diabetes status), outcome assessment, and completeness of follow-up. Our primary outcome was survival. Estimates of treatment effect (single versus bilateral) expressed as hazard ratios were pooled across studies.

FINDINGS

None of the studies was a randomised trial, but nine cohort studies met our inclusion criteria. Seven studies yielded survival data for meta-analysis, and included 15962 patients: 11269 single and 4693 bilateral IMA grafts. The bilateral group had significantly better survival than the single group (hazard ratio for death 0.81; 95% CI 0.70-0.94). Exclusion of methodologically weak studies improved survival rates with bilateral IMA grafts.

INTERPRETATION

Because no study was a randomised trial, our results are more uncertain than is indicated by the 95% CI. Nevertheless, bilateral IMA grafts seem to give better survival rates than single grafts.

A prognostic model for ovarian cancer

About 6000 women in the United Kingdom develop ovarian cancer each year and about two-thirds of the women will die from the disease. Establishing the prognosis of a woman with ovarian cancer is an important part of her evaluation and treatment. Prognostic models and indices in ovarian cancer should be developed using large databases and, ideally, with complete information on both prognostic indicators and long-term outcome. We developed a prognostic model using Cox regression and multiple imputation from 1189 primary cases of epithelial ovarian cancer (with median follow-up of 4.6 years). We found that the significant (P< or = 0.05) prognostic factors for overall survival were age at diagnosis, FIGO stage, grade of tumour, histology (mixed mesodermal, clear cell and endometrioid versus serous papillary), the presence or absence of ascites, albumin, alkaline phosphatase, performance status on the ZUBROD-ECOG-WHO scale, and debulking of the tumour. This model is consistent with other models in the ovarian cancer literature; it has better predictive ability and, after simplification and validation, could be used in clinical practice.

The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials

To comprehend the results of a randomized, controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through complete transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised CONSORT statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement. The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Discussion. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting the information is associated with biased estimates of treatment effect or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly includes the number of participants, for each intervention group, that are included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have performed an intention-to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.

Monitoring of large randomised clinical trials: a new approach with Bayesian methods

BACKGROUND

In judging whether or not to continue enrolling patients into a randomised clinical trial, most data-monitoring and ethics committees (DMECs) rely on the p value for the difference in effect between the study groups. In the 1990s, two randomised controlled trials-one in patients with lung cancer and one in those with head and neck cancer-were instead monitored by Bayesian methods. We assessed the value of this approach in the monitoring of these clinical trials.

METHODS

Before the trials opened, participating clinicians were asked their opinions on the expected difference between the study treatment (continuous hyperfractionated accelerated radiotherapy [CHART]) and conventional radiotherapy. These opinions were used to form an "enthusiastic" and a "sceptical" prior distribution. These prior distributions were combined with the trial data at each of the annual DMEC meetings. If, during monitoring, a result in favour of CHART was seen, the DMEC was to decide whether the results were sufficiently convincing to persuade a sceptic that CHART was worthwhile. Conversely, if there was apparently no or little difference, the DMEC was asked whether they thought the results sufficiently convincing to persuade an enthusiast that CHART was not worthwhile.

FINDINGS

At each of the annual meetings, the DMEC concluded that there was insufficient evidence to convert either sceptics or enthusiasts, and that the trials should therefore remain open to recruitment. Neither trial was closed to recruitment earlier than planned. However if a conventional (p-value-based) stopping rule had been used, the lung-cancer trial would probably have been stopped.

INTERPRETATION

This Bayesian approach to monitoring is simple to implement and straightforward for members of the DMEC to understand. In our opinion, it is more intuitively appealing than conventional approaches.

The efficacy of problem-solving treatments after deliberate self-harm: meta-analysis of randomized controlled trials with respect to depression, hopelessness and improvement in problems

BACKGROUND

Brief problem-solving therapy is regarded as a pragmatic treatment for deliberate self-harm (DSH) patients. A recent meta-analysis of randomized controlled trials (RCTs) evaluating this approach indicated a trend towards reduced repetition of DSH but the pooled odds ratio was not statistically significant. We have now examined other important outcomes using this procedure, namely depression, hopelessness and improvement in problems.

METHOD

Six trials in which problem-solving therapy was compared with control treatment were identified from an extensive literature review of RCTs of treatments for DSH patients. Data concerning depression, hopelessness and improvement in problems were extracted. Where relevant statistical data (e.g. standard deviations) were missing these were imputed using various statistical methods. Results were pooled using meta-analytical procedures.

RESULTS

At follow-up, patients who were offered problem-solving therapy had significantly greater improvement in scores for depression (standardized mean difference = -0.36; 95% CI -0.61 to -0.11) and hopelessness (weighted mean difference =-3.2; 95% CI -4.0 to -2.41), and significantly more reported improvement in their problems (odds ratio = 2.31; 95% CI 1.29 to 4.13), than patients who were in the control treatment groups.

CONCLUSIONS

Problem-solving therapy for DSH patients appears to produce better results than control treatment with regard to improvement in depression, hopelessness and problems. It is desirable that this finding is confirmed in a large trial, which will also allow adequate testing of the impact of this treatment on repetition of DSH.

The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials

To comprehend the results of a randomized controlled trial (RCT), readers must understand its design, conduct, analysis and interpretation. That goal can only be achieved through complete transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised CONSORT statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement.The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results and Discussion. The revised checklist includes 22-items selected because empirical evidence indicates that not reporting the information is associated with biasedestimates of treatment effect or the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of a trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly includes the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have performed an intention-to-treat analysis.In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.

The revised CONSORT statement for reporting randomized trials: explanation and elaboration

Overwhelming evidence now indicates that the quality of reporting of randomized, controlled trials (RCTs) is less than optimal. Recent methodologic analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects. Such systematic error is seriously damaging to RCTs, which boast the elimination of systematic error as their primary hallmark. Systematic error in RCTs reflects poor science, and poor science threatens proper ethical standards. A group of scientists and editors developed the CONSORT (Con solidated S tandards o f R eporting T rials) statement to improve the quality of reporting of RCTs. The statement consists of a checklist and flow diagram that authors can use for reporting an RCT. Many leading medical journals and major international editorial groups have adopted the CONSORT statement. The CONSORT statement facilitates critical appraisal and interpretation of RCTs by providing guidance to authors about how to improve the reporting of their trials. This explanatory and elaboration document is intended to enhance the use, understanding, and dissemination of the CONSORT statement. The meaning and rationale for each checklist item are presented. For most items, at least one published example of good reporting and, where possible, references to relevant empirical studies are provided. Several examples of flow diagrams are included. The CONSORT statement, this explanatory and elaboration document, and the associated Web site ( http://www.consort-statement.org ) should be helpful resources to improve reporting of randomized trials. Throughout the text, terms marked with an asterisk are defined at end of text.

The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials

To comprehend the results of a randomized, controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through complete transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Con solidated S tandards o f R eporting T rials) statement to help authors improve reporting by using a checklist and flow diagram. The revised CONSORT statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement. The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Discussion. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting the information is associated with biased estimates of treatment effect or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of a trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly includes the number of participants, for each intervention group, that are included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have performed an intention-to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.

The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials

To comprehend the results of a randomised controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through total transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by use of a checklist and flow diagram. The revised CONSORT statement presented here incorporates new evidence and addresses some criticisms of the original statement. The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Discussion. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting this information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of a trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly shows the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have done an intention-to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.

The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials

To comprehend the results of a randomised controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through total transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by use of a checklist and flow diagram. The revised CONSORT statement presented here incorporates new evidence and addresses some criticisms of the original statement. The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Discussion. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting this information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from four stages of a trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly shows the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have done an intention-to-treat analysis. In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.