Type-II error rates (beta errors) of randomized trials in orthopaedic trauma

The misuse of 'no significant difference' in British orthopaedic literature

INTRODUCTION

Many studies published in medical journals do not consider the statistical power required to detect a meaningful difference between study groups. As a result, these studies are often underpowered: the sample size may not be large enough to pick up a statistically significant difference (or other effect of interest) of a given size between the study groups. Therefore, the conclusion that there is no statistically significant difference between groups cannot be made unless a study has been shown to have sufficient power. The aim of this study was to establish the prevalence of negative studies with inadequate statistical power in British journals to which orthopaedic surgeons regularly submit.

MATERIALS AND METHODS

We assessed all papers in the last consecutive six issues prior to the start of the study (April 2005) in The Journal of Bone and Joint Surgery (British), Injury, and Annals of the Royal College of Surgeons of England. We sought published evidence that a power analysis had been performed in association with the main hypothesis of the paper.

RESULTS

There were a total of 170 papers in which a statistical comparison of two or more groups was undertaken. Of these 170 papers, 49 (28.8%) stated as their primary conclusion that there was no statistically significant difference between the groups studied. Of these 49 papers, only 3 (6.1%) had performed a power analysis demonstrating adequate sample size.

CONCLUSIONS

These results demonstrate that the majority of negative studies in the British orthopaedic literature that we have looked at have not performed the statistical analysis necessary to reach their stated conclusions. In order to remedy this, we recommend that the journals sampled include the following guidance in their instructions to authors: the statement 'no statistically significant difference was found between study groups' should be accompanied by the results of a power analysis.

Clinical trial design in fracture-healing research: meeting the challenge

The rapidly growing global burden of road-traffic accidents and fragility fractures makes research on fracture repair a vital component of the efforts needed to face this rising public health challenge. The focus on developing new and innovative strategies to treat fractures is easily justifiable given the potential human benefit from such discoveries. Randomized trials remain the standard to which the evaluation of novel fracture-healing therapies must continue to evolve. This article reviews randomized controlled trials in the context of the hierarchy of evidence, special challenges to their conduct in the setting of surgical research, and lessons learned from fracture-healing trials published to date. Suggestions are made regarding the optimal characteristics of fracture models and logistical consideration for ensuring the success of future trials. The realization that surgical trials have unique methodological and interpretative challenges has fueled a renewed vision of the design and execution of large, definitive clinical trials with a meaningful impact on the lives of patients.

The orthopaedic trauma literature: an evaluation of statistically significant findings in orthopaedic trauma randomized trials

Background

Evidence-based medicine posits that health care research is founded upon clinically important differences in patient centered outcomes. Statistically significant differences between two treatments may not necessarily reflect a clinically important difference. We aimed to quantify the sample sizes and magnitude of treatment effects in a review of orthopaedic randomized trials with statistically significant findings.

Methods

We conducted a comprehensive search (PubMed, Cochrane) for all randomized controlled trials between 1/1/95 to 12/31/04. Eligible studies include those that focused upon orthopaedic trauma. Baseline characteristics and treatment effects were abstracted by two reviewers. Briefly, for continuous outcome measures (ie functional scores), we calculated effect sizes (mean difference/standard deviation). Dichotomous variables (ie infection, nonunion) were summarized as absolute risk differences and relative risk reductions (RRR). Effect sizes >0.80 and RRRs>50% were defined as large effects.

Using regression analysis we examined the association between the total number of outcome events and treatment effect (dichotomous outcomes).

Results

Our search yielded 433 randomized controlled trials (RCTs), of which 76 RCTs with statistically significant findings on 184 outcomes (122 continuous/62 dichotomous outcomes) met study eligibility criteria. The mean effect size across studies with continuous outcome variables was 1.7 (95% confidence interval: 1.43–1.97). For dichotomous outcomes, the mean risk difference was 30% (95%confidence interval:24%–36%) and the mean relative risk reduction was 61% (95% confidence interval: 55%–66%; range: 0%–97%). Fewer numbers of total outcome events in studies was strongly correlated with increasing magnitude of the treatment effect (Pearson's R = -0.70, p < 0.01). When adjusted for sample size, the number of outcome events revealed an independent association with the size of the treatment effect (Odds ratio = 50, 95% confidence interval: 3.0–1000, p = 0.006).

Conclusion

Our review suggests that statistically significant results in orthopaedic trials have the following implications-1) On average large risk reductions are reported 2) Large treatment effects (>50% relative risk reduction) are correlated with few number of total outcome events. Readers should interpret the results of such small trials with these issues in mind.

Does delay matter? The restoration of objectively measured shoulder strength and patient-oriented outcome after immediate fixation versus delayed reconstruction of displaced midshaft fractures of the clavicle

Outcome after surgical treatment for nonunion and malunion of midshaft displaced clavicle fractures has generally been described as favorable and equal to results of acute repair. This assumption has been based on subjective criteria, however, and no direct comparison is available in the literature. This study used objective measurements of limb function to compare outcome in patients who underwent delayed operative intervention for nonunion and malunion with the outcome of patients who underwent immediate open reduction and internal fixation after displaced clavicle fracture. All patients had sustained completely displaced, closed, isolated midshaft clavicle fractures, of whom 15 had undergone acute open reduction and internal fixation with a compression plate at a mean of 0.6 months after injury (acute group). Another 15 patients had undergone delayed reconstruction with open reduction, bone grafting, and compression plate fixation for nonunion or malunion a mean of 63 months after injury (delayed group). The 2 groups were similar in age, gender, original fracture characteristics, and mechanism of injury. Complete assessment included standard history and physical examination, the Disabilities of the Arm, Shoulder and Hand (DASH) score and Constant Shoulder Score, subjective rating of outcome satisfaction, and objective muscle strength testing using a previously validated and published protocol on the Baltimore Therapeutic Equipment (BTE) work simulator. There were no significant differences between acute fixation and delayed reconstruction groups with regard to strength of shoulder flexion (acute, 94%; delayed, 93%; P = .82), shoulder abduction (acute, 97%; delayed, 97%; P = .92), external rotation (acute, 97%; delayed, 90%; P = .11), or internal rotation (acute, 98%; delayed, 96%; P = .55). Constant scores in the acute group were superior (acute, 95; delayed, 89; P = .02), but differences in DASH scores were not significant (acute, 3.0; delayed, 7.2; P = .15). Shoulder flexion muscle endurance was significantly decreased in the delayed group (acute, 109%; delayed, 80%; P = .05). Differences in muscle endurance in other planes were not significantly different (abduction endurance: acute, 107%; delayed, 81%; P = .24). Both groups rated their satisfaction with the procedure as excellent. Late reconstruction of nonunion and malunion after displaced midshaft fractures of the clavicle is a reliable and reproducible procedure that results in restoration of objective muscle strength similar to that seen with immediate fixation; however, there are subtle decreases in endurance strength and outcome compared with acute fracture repair. This information should not be used to justify primary operative repair in isolation but is useful in decision-making when counseling patients with displaced midshaft fractures of the clavicle.

Hamstring tendon autograft better than bone patellar-tendon bone autograft in ACL reconstruction: a cumulative meta-analysis and clinically relevant sensitivity analysis applied to a previously published analysis

BACKGROUND

Current debate on treatment options for anterior cruciate ligament (ACL) reconstruction complicate the choice between hamstring and bone-patellar tendon-bone autografts. We hypothesized a priori that cumulative meta-analysis (a form of sensitivity analysis) might show that the evidence for reduction of morbidity by hamstring grafts could have been reached at an earlier time. Furthermore, we hypothesized a priori that modern state-of-the-art hamstring graft fixation technique would give similar results regarding stability as bone-patellar tendon-bone autografts.

METHODS

We performed a cumulative meta-analysis and sensitivity analysis based on femoral graft fixation techniques to compare hamstring autograft and bone-patellar tendon-bone autografts in ACL reconstruction derived from a previously published meta-analysis.

RESULTS

Cumulatively, that hamstring autograft reduces anterior knee pain had already reached statistical significance in 2001 (relative risk 0.49 (95%CI: 0.32-0.76; p = 0.001, I2 = 0%)). The modern endobutton hamstring graft fixation technique (2 studies) yielded similar stability in the Lachman test as bone-patellar tendon-bone grafts, with a relative risk of 1.1 (95%CI: 0.82-1.5; p = 0.6, I2 = 0%). Exclusion of the endobutton group explains the increased laxity in the hamstring graft group.

INTERPRETATION

Cumulative meta-analysis strengthens the evidence for reduced morbidity using hamstring tendon autograft for anterior cruciate ligament reconstruction. Sensitivity analysis focusing on state-of-the-art hamstring graft fixation techniques further weakens the evidence that bone-patellar tendon-bone autografts provide better stability.

Industry-funded positive studies not associated with better design or larger size

Previous studies have associated commercial funding with positive outcomes in orthopaedic research. Those reports, however, failed to account for potential confounding variables that can lead to a disproportion of positive outcomes, including sample size, study design, and study quality. We tested the hypothesis that nonscientific factors (funding source, orthopaedic subspecialty, and geographic region of origin) are associated with positive study outcomes, but not the result of differences in study design, study quality, or sample size. All 747 abstracts presented at the 2004 American Academy of Orthopaedic Surgeons annual meeting underwent blinded analysis using previously published criteria. Studies that received commercial funding were more likely to conclude with positive outcomes. Subspecialty and country of origin were not associated with positive outcomes. Commercially funded studies were not more likely than non-funded studies to be well-designed. When control groups were used, those in commercially funded studies were not larger than those used in nonfunded studies. Our data suggest commercial funding was associated with positive outcomes, but we found no evidence to suggest commercially funded studies were better designed or larger than non-funded studies.

Outcome measurements in orthopedic

The choice of outcome measure in orthopedic clinical research studies is paramount. The primary outcome measure for a study has several implications for the design and conduct of the study. These include: 1) sample size determination, 2) internal validity, 3) compliance and 4) cost. A thorough knowledge of outcome measures in orthopedic research is paramount to the conduct of a quality study. The decision to choose a continuous versus dichotomous outcome has important implications for sample size. However, regardless of the type of outcome, investigators should always use the most 'patient-important' outcome and limit bias in its determination.

Early mobilization in a removable cast compared with immobilization in a cast after operative treatment of ankle fractures: a prospective randomized study

BACKGROUND

The aim of operative treatment for ankle fractures is to allow early movement after internal fixation. The hypothesis of this study was that early mobilization would improve functional recovery in patients after surgery for ankle fractures.

METHODS

In a prospective randomized controlled study, 66 consecutive patients with ankle fractures that required open reduction and internal fixation (ORIF) were assigned to one of two postoperative regimens: immobilization in a nonweightbearing below-knee cast or early mobilization in a removable cast. Four patients were excluded from the study, leaving 62 for review.

RESULTS

Patients who had early mobilization in a removable cast had higher functional scores (Olerud-Molander and AOFAS) at 9 and 12 weeks postoperatively. They also returned to work earlier (67 days) compared with those treated in nonweightbearing below-knee cast (95 days), p<0.05. There was no statistical difference in Quality of Life (SF-36 Questionnaire) at 6 months between the two groups. We had an approximately 10% postoperative infection trend (one superficial and two deep) in the early mobilization group.

CONCLUSION

Despite the overall short-term benefit of early mobilization, we had three patients in the early mobilization group who had wound complications. Both the surgeon and patient should be aware of the higher risk of wound complications associated with this treatment, and thus the accelerated rehabilitation protocol should be individualized.

Malunions of the finger metacarpals and phalanges

Many malunions of the finger metacarpals are mild and do not require or justify operative intervention. Early recreation of the fracture or osteotomy is more likely to be rewarded with favorable results than late operation. Rotational malunions of the metacarpals or proximal phalanges may be treated by transverse extra-articular transverse or step-cut osteotomies at or proximal to the malunion site. Rotational malunions of the proximal phalanges as great as 200 in the index, middle, and ring fingers and 300 in the small finger may be managed by transverse extra-articular osteotomy at the adjoining metacarpal base. Angular and combined angular and rotational deformities of the metacarpal can be corrected by closing wedge osteotomy at the malunion site, with adjustment for malrotation when necessary. Angular and combined angular and rotational deformities of the proximal phalanx may be corrected by dorsal opening or lateral opening or closing wedge osteotomy, with derotation when needed. Articular malunions may be treated by osteotomy at the fracture site, a sliding osteotomy of the fracture and its proximal supporting cortex, or extra-articular osteotomy. Each approach for articular malunions has its potential risks and benefits. The true risks of articular malunion correction may not be fully known, because of the small number of cases in each presented series and the short follow-ups. Finger motion may be improved by correction of deformity alone, and may be further enhanced by tenolysis of adjacent adhesions. Capsulolysis may be helpful in instances of adjacent joint contracture. Despite improvement of finger motion in a majority of cases, some degree of remaining stiffness is common. Stiffness is almost always a residual of the original injury rather than a complication of corrective surgery, and serves to reinforce the fact that primary fracture reduction, stabilization, and rehabilitation are usually the best deterrents to malunion and consequent impairment. Much of the best available information has been gained from retrospective cohort or case study reports that may have inherent flaws in study design that limit their statistical validity and ability to detect trends. Flaws may include heterogeneity; investigator enthusiasm; and a lack of enrollment, prospective controlled randomization, blinding, confidence interval determinations, and follow-up. The statistical ability to determine trends in past reports may be compromised. Past reports provide important information and advances, but should be interpreted with some discretion. The pen may be mightier than the scalpel. In spite of encouraging reported results, phalangeal and articular osteotomies, in particular, remain daunting procedures for most hand surgeons. Prospective, controlled randomized studies maybe difficult to achieve in the clinical setting because of the time that would be necessary to secure adequate enrollments for statistical validity andthe occurrence of "dropouts" before completion of adequate follow-up. Meta-analysis is difficult because of variations in discriminators for patient selection and clinical outcomes. Although multicenter studies have their own inherent flaws, they may represent the best future option to add a higher level of study design and validity as compared with past studies. The incorporation of subjective patient outcome instruments into future studies might also provide valuable information. Investigators should review previous reports with a goal of improving study designs and scientific methodology, confirming or contradicting past results, or adding new information.

Critical evaluation of scientific articles and other sources of information: An introduction to evidence-based veterinary medicine

The purpose of this paper is to briefly review key concepts regarding critical reading of the scientific literature to make informed decisions, in the context of evidence-based veterinary medicine. Key concepts are reviewed, based on the broader experience in human medicine, with adaptations, as indicated, to veterinary medicine. That a paper has been published in a peer-reviewed journal does not guarantee its credibility; guidelines are given regarding the general merit of different kinds of articles, as well as checklists and criteria that can be used to assess a paper. Specific study designs, their merits and limitations, are briefly discussed. Standard numerical indices for assessment of studies involving treatments and for assessments of diagnostic tests are summarized. Criteria for assessing drug trials are presented. The principles of statistical analysis are described, including practical considerations and common errors. Finally, numerous sources of bias are reviewed.

Common errors in the design of orthopaedic studies

There are several issues that one has to take into consideration in order to avoid potential pitfalls in the design of orthopaedic studies. This article highlights how to avoid common errors and how to continue the drive towards the unattainable, but laudable, goal of perfection.

Evidence-based medicine: what it is and what it is not

The evidence-based medicine (EBM) related concepts of hierarchy of evidence, meta-analyses, confidence intervals, study design, etc. are nowadays so widespread, that clinicians willing to use today's medical literature with understanding have no choice but to become familiar with EBM principles and methodologies. Although surgeons may perceive that evidence-based medicine mandates a strict adherence to randomised trials, it more accurately involves informed and effective use of all types of evidence (from meta-analysis of randomised trials to individual case series and case reports. With the ever-increasing amount of available information, surgeons must consider a shift in paradigm from traditional practice to one that involves question formulation, validity assessment of available studies and appropriate application of research evidence to individual patients.

Common pitfalls in the conduct of clinical research

Recently, paradigm shift from expert opinion towards evidence-based medicine has occurred encouraging physicians to base their treatment decisions on the best available research evidence. In the hierarchy of evidence randomized clinical trials (level 1 evidence) are considered of the highest quality (least biased) while non-randomized studies represent lower levels (levels 2-4). Several pitfalls in the design and conduct of clinical research include: lack of randomization, lack of concealment, lack of blinding, and errors in hypothesis testing (type I and II errors). A basic understanding of these principles of research will empower both investigators and readers when applying the results of research to clinical practice.

Evidence-Based Medicine and Clinical Trials in Pain Practice and Orthopedics

Medical practices should be based on scientific findings pursuant to the rules of evidence-based medicine. Quality standards for interventional pain therapy and orthopedic clinical studies have been lacking. As a result, the efficacy of many forms of therapy is insufficiently documented, making the level of evidence low. This article identifies common deficiencies in the conduct of clinical trials, as well as limitations in conducting randomized controlled studies. Recommendations for improvement are provided. The discussion provides the clinically active physician with interpretation aids for the evaluation of meta-analyses, supports personal evidence-based decisions, and reviews the most important principles for planning and conducting of experimental clinical studies. Current examples in the literature verify the implementation of these principles and present current findings in accordance with evidence-based medicine (EBM) criteria. In spite of an increasing emergence of EBM-based studies, we conclude that the number of well-designed, high quality, controlled studies conducted in accordance with the guidelines of Good Clinical Practice examining interventional pain therapy and orthopedic clinical studies remains unacceptably low.

Biomechanics and biology of plate fixation of distal radius fractures

The fracture management principles of anatomic or near anatomic reduction, fracture stabilization, minimal operative trauma, and early joint motion are paramount in man-aging unstable distal radial fractures. The operative approach and plate selection should correlate with the fracture configuration. Plates have the advantages of providing secure fixation throughout the entire healing process without protruding wires or pins and allowing early and intensive forearm, wrist, and digital exercises. Disadvantages include additional operative trauma, including fragment devascularization; some additional risk of wrist stiffness; occasional tendon rupture; and at times, the need for plate removal. New developments in plate and screw design and operative strategies, fragment specific fixation, and plate strength have improved results with plate fixation. Fixed angle blades and locking screws and pegs enhance overall plate stability, support the articular surface of the distal radius, and are effective in fractures occurring in osteopenic bone.

External fixation versus locked intramedullary nailing in tibial shaft fractures: a prospective, randomised study of 78 patients

INTRODUCTION

We performed a prospective, randomised study to compare the Ex-fi-re external fixator (EF) with locked intramedullary (IM) nailing in tibial fractures. Only fractures without soft-tissue problems of importance were included.

MATERIALS AND METHODS

Ex-fi-re is a unilateral, dynamic axial fixator with fracture reduction capabilities. The Grosse-Kempf nail was used for nailing. A total of 78 patients with 79 fractures were entered in the study (41 Ex-fi-re, 38 IM nails).

RESULTS

Time to radiographic union and full weight-bearing did not differ significantly, but unprotected weight-bearing was achieved earlier in the IM group (12 vs 20 weeks; p<0.001). There were more reoperations due to secondary dislocation in the EF group. There were no differences in final angulation or shortening. After 6 months and 1 year there were no differences in knee motion, ankle motion, fracture site pain or ankle pain. Some 64% of the nailed patients complained of anterior knee pain after 1 year.

CONCLUSION

The results were comparable in most respects. Unprotected weight-bearing was achieved earlier after IM nailing. Anterior knee pain was frequent after nailing.

Decision analysis in orthopaedics

Orthopaedic surgeons are faced with an ever-growing amount of clinical information from which they are required to make treatment decisions. Many of these decisions can be approached with relative certainty. However, there are many situations where the optimal decision is less clear. These treatment decisions will have competing risks, benefits, or costs. Decision analysis is one method to critically evaluate alternative treatment options with multiple potential outcomes. This method of decision making can be extremely valuable because of the growing number of treatment alternatives, and to the ever-increasing complexity of medical scenarios.

Type II error in the spine surgical literature

STUDY DESIGN

A literature review.

OBJECTIVES

To determine the frequency of potential type II errors published in the spine surgical literature.

SUMMARY OF BACKGROUND DATA

The randomized controlled trial is the strongest clinical evidence available in investigational medicine. Unfortunately, it is common for randomized controlled trials published in peer-reviewed journals not to report a primary question or a sample size calculation. When the null hypothesis is accepted and the power of a study is unreported, the validity of a study's findings may be significantly limited. To our knowledge, the spine literature has not been appraised to determine the frequency of type II errors. METHODS.: A literature search was conducted of MED-LINE, PubMed, and Cochrane databases, using the key words of "spine" and "surgery" between 1967 and 2002. Trials were included if they were of a 2-group randomized controlled trial design, which reported a nonsignificant difference in the primary outcome. The frequency of reporting the primary outcome and sample size calculation was determined. The sample size was assessed to determine whether the trial had sufficient patients to detect a 10%, 25%, and 35% relative difference in the primary outcome for a power of 80%.

RESULTS

A total of 37 studies satisfied the inclusion criteria. Six studies reported a sample size calculation (17%). Of the remaining 31 studies, 5 explicitly stated a primary outcome (14%). The mean type II error (beta error) was 82%.

CONCLUSION

The spine surgical literature is plagued with a high potential for type II error. A trial's methodology should be scrutinized to prevent misinterpretation of the results.

Methodologic issues in systematic reviews and meta-analyses

Systematic reviews of original research are increasing in number. Systematic reviews are distinct from narrative reviews because they address a specific clinical question, require a comprehensive literature search, use explicit selection criteria to identify relevant studies, assess the methodologic quality of included studies, explore differences among study results, and either qualitatively or quantitatively synthesize study results. Systematic reviews that quantitatively pool results of more than one study are called meta-analyses. Several organizations are collaboratively involved in producing high quality systematic reviews and meta-analyses. Familiarity with how to do a systematic review and meta-analysis will lead to greater skill in using this type of article. For clinicians, teachers, and investigators, systematic reviews and meta-analyses are useful sources of evidence.

Common statistical methods in orthopaedic clinical studies

We discuss the statistical representation and management of random error in orthopaedic clinical studies. Descriptive studies (such as case series) collect information about a sample that may be generalized to describe a population. Typically this description is in the form of summary statistics, such as means, proportions, or rates. Error in these variables may be represented by confidence intervals. Correlation and regression are techniques for investigation of the relationship between two or more variables. Descriptive statistics, comparisons of groups, especially hypothesis tests, and assessment of association including correlation and regression are important statistical concepts for clinicians involved in the conduct or appraisal of orthopaedic clinical research.

Statistical sampling and hypothesis testing in orthopaedic research

The purpose of the current article was to review the process of hypothesis testing and statistical sampling and empower readers to critically appraise the literature. When the p value of a study lies above the alpha threshold, the results are said to be not statistically significant. It is possible, however, that real differences do exist, but the study was insufficiently powerful to detect them. In that case, the conclusion that two groups are equivalent is wrong. The probability of this mistake, the Type II error, is given by the beta statistic. The complement of beta, or 1-beta, representing the chance of avoiding a Type II error, is termed the statistical power of the study. We previously examined the statistical power and sample size in all of the studies published in 1997 in the American and British volumes of the Journal of Bone and Joint Surgery, and in Clinical Orthopaedics and Related Research. In the journals examined, only 3% of studies had adequate statistical power to detect a small effect size in this sample. In addition, a study examining only randomized control trials in these journals showed that none of 25 randomized control trials had adequate statistical power to detect a small effect size. However, beta, or power, is less well understood. Because of this, researchers and readers should be aware of the need to address issues of statistical power before a study begins and be cautious of studies that conclude that no difference exists between groups.

Underpowering in randomized trials reporting a sample size calculation

OBJECTIVE

The objective of this study was to determine whether standard deviations (SDs) used in sample size calculations are smaller than those found in the resulting study sample, thereby leading to underpowered studies.

METHOD

The predicted SD used in the sample size calculation and the actual SD of the study sample were recorded for randomized trials recently published in one of four major journals.

RESULTS

Sample SD was greater than predicted SD for 80% of endpoints. About one quarter of trials required five times as many patients as specified in the sample size calculation.

CONCLUSION

Trials reporting sample size calculations for continuous endpoints published in the most reputable medical journals are often underpowered. There seems to be insufficient understanding that the SD of a sample of patients is a random variable, associated with imprecision, that cannot easily be extrapolated from one population to another.

The risk of false-positive results in orthopaedic surgical trials

The risk of concluding that the results of a particular study are true, when, in fact, they really are attributable to chance (or random sampling error) is underappreciated by investigators. This erroneous false-positive conclusion is designated as a Type I or alpha error. The extent to which randomized trials in surgery risk Type I errors is unclear. The current authors hand-searched four orthopaedic journals, six general surgery journals, and five medical journals to identify recently published randomized trials (within the past 2 years). Information on outcomes and statistical adjustment for multiple outcomes was recorded for each study. The risk of a Type I error was calculated for each study that did not explicitly state a primary outcome measure for the main statistical comparison. One hundred fifty-nine studies met the inclusion criteria for the study: 60 studies from orthopaedic journals, 49 studies from nonorthopaedic surgical journals, and 50 studies from medical journals. Of the trials that did not state a primary outcome measure, the risk of Type I errors (false-positive results) in orthopaedic and nonorthopaedic surgery journals (mean 37.3% +/- 13.3% and 37.6% +/- 10.5%, respectively) were significantly greater than medical journals (10.1% +/- 1.9%). In the current review of randomized trials in surgery and medicine, the following is reported: (1) reporting of primary outcomes in trials was inadequate; (2) one in three trials in surgery and one in 10 trials in medicine risked false-positive results; and (3) few trials in surgery and medicine considered adjustment for multiple comparisons.

The validity of claims made in orthopaedic print advertisements

PURPOSE

Orthopaedic surgeons are frequently presented with advertisements for orthopaedic and medical products in which companies make claims of clinical and scientific fact. This study was designed to evaluate the statements made in orthopaedic print advertisements and determine whether they are supported by scientific data.

METHODS

Fifty statements from fifty advertisements were chosen at random from six peer-reviewed orthopaedic journals. The companies that placed the advertisements were contacted to provide supporting data for the statement of clinical or scientific fact. Three senior orthopaedic surgeons evaluated the data for quality and support. A high-quality study was defined as a study that could be published in the peer-reviewed literature. A well-supported statement was defined as a statement with enough supporting evidence to be used in clinical practice. The evaluating surgeons were blinded to product and company identification.

RESULTS

The supporting data were from a published source for eighteen claims (36%), from a presentation at a public forum or a scientific meeting for twelve claims (24%), or were "data on file" only at the company for twelve claims (24%). Interobserver agreement among the surgeons evaluating the advertisements for quality and support was good (the average intraclass correlation coefficient was 0.72). Of the fifty claims, twenty-two were considered unsupported by scientific data, seventeen were classified as possibly supported, seven were well supported, and four were from companies that did not respond despite three requests. Claims that were supported by published data were significantly more likely to be rated as well supported (p < 0.001). All twelve claims that were supported purely by "data on file" at the company were considered to be poorly supported.

CONCLUSIONS

Orthopaedic surgeons should interpret claims made in orthopaedic print advertisements with caution. Approximately half of the claims are not supported by enough data to be used in a clinical decision-making process.

Autologous blood transfusion after total knee arthroplasty: a randomized, prospective study comparing predonated and postoperative salvage blood

A randomized, prospective study of the use of allogeneic blood was performed in a consecutive series of patients who underwent primary total knee arthroplasty (TKA) and had autologous transfusion either from one unit of predonated autologous blood or from postoperative unwashed blood salvage. In this study, 83 patients (88 knees) were included, with 47 knees in the salvage group and 41 in the predonation group. There were no differences between groups in average age, height, and weight, or gender, diagnoses, or anesthesia type. No significant difference was seen between the groups in the prevalence of allogeneic blood transfusion (5% for the predonation group and 0% for salvage group). Postoperative blood salvage was as effective as predonated autologous blood in preventing the risk associated with allogeneic blood after TKA.

Use of a cast compared with a functional ankle brace after operative treatment of an ankle fracture. A prospective, randomized study

BACKGROUND

Controversy continues with regard to the optimal postoperative care after open reduction and internal fixation of an ankle fracture. The hypothesis of this study was that postoperative treatment of an ankle fracture with a brace that allows active and passive range-of-motion exercises would improve the functional recovery of patients compared with that after conventional treatment with a cast. Thus, the purpose of this prospective, randomized study was to compare the long-term subjective, objective, and functional outcome after conventional treatment with a cast and that after use of functional bracing in the first six weeks following internal fixation of an ankle fracture.

METHODS

One hundred patients with an unstable and/or displaced Weber type-A or B ankle fracture were treated operatively and then were randomly allocated to two groups: immobilization in a below-the-knee cast (fifty patients) or early mobilization in a functional ankle brace (fifty patients) for the first six postoperative weeks. The follow-up examinations, which consisted of subjective and objective (clinical, radiographic, and functional) evaluations, were performed at two, six, twelve, and fifty-two weeks and at two years postoperatively.

RESULTS

There were no perioperative complications in either study group, but eight patients who were managed with a cast and thirty-three patients who were managed with a brace had postoperative complications, which were mainly related to wound-healing. Two patients in the group treated with a cast had deep-vein thrombosis. All fractures healed well in both groups. The difference between the two groups with respect to the complication rate was significant (p = 0.0005). No significant differences between the study groups were observed in the final subjective or objective (clinical) evaluation. At the two-year follow-up examination, the average score (and standard deviation) according to the ankle-rating scale of Kaikkonen et al. was 85 +/- 9 points for the group treated with a cast and 83 +/- 10 points for the group treated with a brace, and the average ankle score according to the system of Olerud and Molander was 87 +/- 8 points and 87 +/- 9 points, respectively.

CONCLUSIONS

The long-term functional outcome after postoperative treatment of an ankle fracture with a cast and that after use of a functional brace are similar. Although early mobilization with use of a functional ankle brace may have some theoretical beneficial effects, the risk of postoperative wound complications associated with this treatment approach is considerably increased compared with that after conventional cast treatment. Thus, the postoperative protocol of treatment with a functional brace requires refinement before it can be generally advocated for use after operative treatment of an ankle fracture.

The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000

BACKGROUND

The purpose of this study was threefold: (1) to determine the scientific quality of published randomized trials in the American Volume of The Journal of Bone and Joint Surgery from 1988 through 2000, (2) to identify predictors of study quality, and (3) to evaluate inter-rater agreement in the scoring of study quality with use of a simple scale.

METHODS

Hand searches of The Journal of Bone and Joint Surgery were conducted in duplicate to identify randomized clinical trials. Of 2468 studies identified, seventy-two (2.9%) met all eligibility criteria. Two investigators each assessed the quality of the study under blinded conditions and abstracted relevant data.

RESULTS

The mean score (and standard error) for the quality of the seventy-two randomized trials was 68.1% plus minus 1.6%; 60% (forty-three) scored <75%. Drug trials had a significantly higher mean quality score than did surgical trials (72.8% compared with 63.9%, p < 0.05). Regression analysis revealed that cited affiliation with an epidemiology department and cited funding were associated with higher quality scores. Failure to conceal randomization, to blind outcome assessors, and to describe why patients were excluded resulted in significantly lower quality scores (p < 0.05), more than the 5% decrease expected by removal of each item. A priori calculations of sample size were rarely performed in the reviewed studies, and only 2% of the studies with negative results included a post hoc power analysis. The Detsky quality scale met accepted standards of interobserver reliability (kappa, 0.87; 95% confidence interval, 0.70 to 0.95).

CONCLUSIONS

Few studies published in The Journal of Bone and Joint Surgery were randomized trials. More than half of the trials were limited by a lack of concealed randomization, lack of blinding of outcome assessors, or failure to report reasons for excluding patients. Application of standardized guidelines for the reporting of clinical trials in orthopaedics should improve quality.