Statistics in medical journals: some recent trends

I review some areas of medical statistics that have gained prominence over the last 5-10 years: meta-analysis, evidence-based medicine, and cluster randomized trials. I then consider several issues relating to data analysis and interpretation, many relating to the use and misuse of hypothesis testing, drawing on recent reviews of the use of statistics in medical journals. I also consider developments in the reporting of research in medical journals.

Readiness to change: newspaper coverage of tobacco farming and diversification

Diversification, like tobacco use prevention and cessation, is an important public health concern. The multilevel patterns of tobacco dependency suggest the need for public health approaches to the "tobacco problem." To understand how newspaper and wire service journalists cover issues involving diversification among tobacco farmers, the authors performed a content analysis of a subset of 100 articles on diversification and tobacco farming. Prochaska and DiClemente's stages of change model was applied to the "problem behavior" of tobacco farming. Among news accounts relating to tobacco farmers or tobacco farming, print media accounts gave relatively little attention to the issue of diversification. Farmers in the sample of news accounts were generally cognizant of pressures to diversify away from reliance on tobacco cultivation but were frustrated due to obstacles to diversification such as limited diversification options and relative absence of infrastructure supports. Community leaders and policy-relevant sources generally supported diversification.

Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery

This paper aims to explore the potential usefulness and limitations of indirect comparisons in evaluating the relative efficacy of interventions. From a systematic review of antimicrobial prophylaxis in colorectal surgery, we identified 11 sets of randomized trials that can be used to compare antibiotics both directly and indirectly. The discrepancy between the direct and the indirect comparison is defined as the absolute value of difference in log odds ratio. The adjusted indirect comparison has the advantages that the prognostic factors of participants in different trials can be partially taken into account and more uncertainty be incorporated into its result by providing a wider confidence interval. However, considerable discrepancies exist between the direct and the adjusted indirect comparisons. When there is no direct comparison, the adjusted indirect method may be used to obtain some evidence about the relative efficacy of competing interventions, although such indirect results should be interpreted with great caution. Further empirical and methodologic research is needed to explore the validity and generalizability of the adjusted indirect comparison for evaluating different interventions.

Adherence in social context

Much has been written about adherence to interventions in older adults. What has not been discussed as extensively is how adherence is influenced and affected by the multiple interacting layers of the social context. Guided by an ecological or multilevel system model, this paper explores how social context may impact adherence. We conclude that when considering strategies to promote adherence to a particular regime, one must take into consideration the interplay between the social context and the adherence-related behavior. Control Clin Trials 2000;21:184S-187S

What do we mean by validating a prognostic model?

Prognostic models are used in medicine for investigating patient outcome in relation to patient and disease characteristics. Such models do not always work well in practice, so it is widely recommended that they need to be validated. The idea of validating a prognostic model is generally taken to mean establishing that it works satisfactorily for patients other than those from whose data it was derived. In this paper we examine what is meant by validation and review why it is necessary. We consider how to validate a model and suggest that it is desirable to consider two rather different aspects - statistical and clinical validity - and examine some general approaches to validation. We illustrate the issues using several case studies.

Tobacco growers' knowledge of revenue distribution and foreign prices: implications for health education

This study examined tobacco growers' knowledge of tobacco economics, particularly their knowledge of how tobacco revenue is distributed and their knowledge of the price of foreign tobacco. Tobacco growers (n = 1236) from 14 tobacco-dependent counties in North Carolina were interviewed by telephone and asked to estimate how much of a hypothetical $2.00 for a pack of cigarettes is received by each of four market segments: tobacco growers, retailers, government and manufacturers. Respondents were also asked to estimate the price of foreign raw tobacco. USDA data were used to assess the accuracy of the respondents' estimates. Respondents were within the 'correct' range as follows: growers 31%, retailers 15%, government 23%, manufacturers 43% and foreign prices 25%. Knowledge was positively related to education and farm size for the growers' share and foreign tobacco prices. Knowledge of the government's and manufacturers' shares was positively associated with younger age. North Carolina growers lacked knowledge of the distribution of the tobacco dollar. This may influence the positions that growers take on health policy and provide significant opportunities for health educators to become involved in grassroots efforts to educate growers.

What do we mean by validating a prognostic model?

Prognostic models are used in medicine for investigating patient outcome in relation to patient and disease characteristics. Such models do not always work well in practice, so it is widely recommended that they need to be validated. The idea of validating a prognostic model is generally taken to mean establishing that it works satisfactorily for patients other than those from whose data it was derived. In this paper we examine what is meant by validation and review why it is necessary. We consider how to validate a model and suggest that it is desirable to consider two rather different aspects - statistical and clinical validity - and examine some general approaches to validation. We illustrate the issues using several case studies.

What do we mean by validating a prognostic model?

Prognostic models are used in medicine for investigating patient outcome in relation to patient and disease characteristics. Such models do not always work well in practice, so it is widely recommended that they need to be validated. The idea of validating a prognostic model is generally taken to mean establishing that it works satisfactorily for patients other than those from whose data it was derived. In this paper we examine what is meant by validation and review why it is necessary. We consider how to validate a model and suggest that it is desirable to consider two rather different aspects - statistical and clinical validity - and examine some general approaches to validation. We illustrate the issues using several case studies.

Teaching non-specialist health care professionals how to identify the atypical mole syndrome phenotype: a multinational study

The atypical mole syndrome (AMS) phenotype is the strongest known risk factor for cutaneous melanoma but recognition of the phenotype has been claimed to be problematic and to require specialist assessment. This study determined the ability of previously unskilled doctors and nurses in five countries to recognize the phenotype after brief training. The system used was the AMS scoring system. This incorporates melanocytic naevus counts, clinical atypia of naevi and distribution of naevi. The agreement in scoring between the dermatologist and trained personnel was determined in 986 patients; overall agreement in diagnosis was 94.5% (kappa 0.70, P < 0.0001). The kappa scores in different countries ranged from 0.65 to 0.77 for individual naevus characteristics, indicative of good agreement. Accurate diagnosis of the atypical mole syndrome phenotype is possible by non-specialists. This has implications for collaborative studies of naevi, for screening and for both primary and secondary prevention of melanoma.

Measuring agreement in method comparison studies

Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference +/- 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.

The Cochrane Collaboration

The Cochrane Collaboration is an international organisation that aims to help people make well-informed decisions about health care by preparing, maintaining and ensuring the accessibility of systematic reviews of the benefits and risks of health care interventions. We describe the structure and activities of the Cochrane Collaboration, especially focusing on the carrying out of systematic reviews of randomised trials. We describe the place of surgery and surgeons within the Cochrane Collaboration.

What randomized trials and systematic reviews can offer decision makers

It is important to recognize the studies that yield the most reliable evidence upon which to base treatment decisions. Randomized trials have a particular place in providing high-quality unbiased comparisons of different treatments, when carried out to a high methodological standard. Empirical evidence shows that such trials are not always done well, and also that poor methodology is associated with biased findings. Consumers of the published literature need to be able to recognize which trials can be trusted. Systematic reviews and meta-analyses offer an organized approach to assessing all the relevant literature on a topic, particularly when several randomized trials address the same treatment comparison. Like trials, systematic reviews and meta-analyses need to be carried out to a high standard. They offer particular potential for providing the most useful information for clinical decision making. Critical aspects of these types of study are considered in this paper.

Risk of bacterial infection associated with allogeneic blood transfusion among patients undergoing hip fracture repair

BACKGROUND

The relationship between allogeneic blood transfusion and bacterial infection remains uncertain. An increased risk of bacterial infection would represent the most important risk of allogeneic transfusion, because viral disease transmission has become so rare.

STUDY DESIGN AND METHODS

A retrospective cohort study of 9598 consecutive hip fracture patients at least 60 years old who underwent surgical repair was performed. The primary outcome was serious bacterial infection, defined as bacteremia, pneumonia, deep wound infection, or septic arthritis or osteomyelitis. Secondary outcomes included two individual infections, pneumonia and urinary tract infection (UTI), and the cost of infection. Hospital cost of infection was assessed by linking the study population to Medicare data.

RESULTS

Fifty-eight percent of patients received at least one transfusion. Serious bacterial infection occurred in 437 patients (4.6%); 28.8 percent of this group died during the hospital stay. Pneumonia occurred in 361 patients (3.8%) and UTI occurred in 1157 patients (12.1%). The adjusted risk of serious bacterial infection associated with transfusion was 1.35 (95% CI, 1.10-1.66). The adjusted risk for pneumonia was 1.52 (95% CI, 1.21-1.91), and that for UTI was 1.03 (95% CI, 0.91-1.17). A dose-response relationship was present for serious bacterial infection (p = 0.001) and pneumonia (p = 0.001). The cost of hospitalization was $14,000 greater for patients with serious infection than for patients without infection.

CONCLUSION

Blood transfusion is associated with a 35-percent greater risk of serious bacterial infection and a 52-percent greater risk of pneumonia. Postoperative infections are costly. The risk of bacterial infection may be the most common life-threatening adverse effect of allogeneic blood transfusion.

Measuring agreement in method comparison studies

Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference +/- 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.