Invention Versus Gold Standard: A Hands-On Research Pearl on Study Design and Statistical Concerns

The Blight of the Type II Error: When No Difference Does Not Mean No Difference

Much focus in research has been given to minimizing type I errors, where we incorrectly conclude that there is a difference between 2 treatments or populations. In contrast, our standard scientific method and power analysis allows for a much greater rate of type II errors, in which we fail to show a difference when, in fact, one exists (≥20% rate of type II errors vs ≤5% rate of type I errors). Additional factors that can cause type II errors may propel their prevalence to well in excess of 20%. Failure to reject the null hypothesis may be a tolerable outcome in a certain proportion of studies. However, type II errors may become dangerous when the conclusions of a study overreach, incorrectly stating that there is no difference, when, in fact, a difference exists. Type II errors resulting in overreaching conclusions may impede incremental advances in our field, as the advantages of small improvements may go undetected. To avert this danger in studies that fail to meet statistical significance, we as researchers (20% or more, vs 5% for type I errors) be precise in our conclusions stating simply that the null hypothesis could not be rejected.

Effect of Vancomycin Soaking on Anterior Cruciate Ligament Graft Biomechanics

PURPOSE

To evaluate the effect of soaking of anterior cruciate ligament (ACL) grafts in vancomycin solution on graft biomechanical properties at the time of implantation.

METHODS

The central third of patellar tendons was harvested from mature bovine knees and prepared as a tendon-only graft or a bone-tendon-bone (BTB) graft. Tendons were wrapped in gauze soaked in vancomycin solution (VS) (5 mg/mL) or normal saline (NS) and left to stand for 30 minutes at room temperature, simulating graft exposure times in the operating room during ACL reconstruction. Tensile testing was carried out on a materials testing system with (1) low-magnitude loading (60 N at 3 mm/s) with repeated testing of tendon-only grafts; and (2) high-magnitude loading (600 N at 10 mm/min) of BTB grafts. For tendon-only grafts, specimens were first wrapped in NS-soaked gauze and underwent testing, with repeated testing performed after wrapping in gauze soaked in VS or buffered VS (pH 7.0). For BTB grafts, specimens were randomly assigned to treatment with VS or NS.

RESULTS

For tendon-only grafts, there was no difference in Young's modulus (YM) after soaking with VS soaking (baseline, 12.69 MPa; treatment, 16.07 ± 4.44 MPa; P = .99) or buffered VS (baseline, 12.45 ± 4.55 MPa; treatment, 15.56 ± 2.83 MPa; P = .99). For BTB grafts, there were no differences in elongation strain (VS, 46.8% ± 7.0%; NS, 31.5% ± 13.5%, P = .19) or YM (VS, 158.4 ± 15.8 MPa; NS, 158.5 ± 23.3 MPa, P = .99).

CONCLUSIONS

According to controlled biomechanical tests, vancomycin soaking of patellar tendon grafts does not adversely affect time-zero material properties.

CLINICAL RELEVANCE

This study suggests that vancomycin wrapping has no immediate adverse effects on the biomechanical properties of ACL grafts. Randomized controlled trials are warranted to validate the widespread use of vancomycin soaking of tendon grafts for infection prophylaxis during ACL reconstruction.

Medical Research Publication: An Insider's Guide

Most original scientific articles submitted to high-impact medical journals are not accepted for publication. Reasons for rejection are diverse, and tips and pearls to improve chances for acceptance are manifold. Four essential points could maximize the chance that submission of a scientific article will result in acceptance and publication. First, before initiation of a study, it is valuable to state a hypothesis detailing what one expects the study to show. Second, the conclusion should be based exclusively on, and not overreach, the results. If researchers start with a hypothesis, describing the conclusion is simple; the results either do, or do not, support the hypothesis. Third, the methods must address the purpose of the study. This sounds obvious, but poorly designed methods can fatally flaw the study, so methods should be written before initiation of a study, and this is the time to seek expert advice regarding whether one's methods could be improved. Fourth, a prospective power analysis will ensure the study includes a sufficient number of patients to avoid failure to detect a difference between study groups due to an insufficient sample size (β error). In summary, before starting a study: state the hypothesis, write the methods, perform a power analysis, and conscientiously review these 3 essentials with both expert mentors and a statistician. Finally, this will mitigate against fatal methodological flaws, and the results of the study will clearly support the study hypothesis-or not-resulting in a definitive conclusion. In the end, authors following these essential guidelines could have improved odds of having their research submissions accepted for publication in a prestigious peer-reviewed medical journal such as Arthroscopy or Arthroscopy, Sports Medicine, and Rehabilitation.

Methods to Improve Arthroscopic and Orthopaedic Biomechanical Investigations: A Few of Our Favorite Things

Controlled biomechanical studies, generally in vitro (and often ex vivo), may represent a first step in evaluation of a new arthroscopic or orthopedic implant or technique. The purpose and methods of biomechanical studies must be thoughtfully considered to achieve results translatable to a clinically relevant conclusion. A limitation is that with the exception of animal studies or rare human investigations, most biomechanical studies actually only investigate mechanics and do not study biological healing. We review tips and pearls for performing quality biomechanical investigations.

Arthroscopy, Sports Medicine, and Rehabilitation (ASMAR): A New, Open Access, Electronic Journal

Arthroscopy, Sports Medicine, and Rehabilitation (ASMAR) is our new, open-access, electronic, companion journal to Arthroscopy and Arthroscopy Techniques. While there is already "so much to read, so little time," bearing in mind the context that some submissions are not accepted for publication in Arthroscopy despite admirable quality, the necessity of ASMAR becomes clear.

Our Measure of Medical Research Should Be Appreciable Benefit to the Patient

The clinical relevance of research is much more important than statistical significance. Patient-reported outcome measures (PROM) are the strongest determinants of satisfaction as the result of an intervention or treatment. Outcomes can be measured in terms of the minimal clinically important difference (MCID) detectable by a patient, bearing in mind that "detectable" includes worsening as well as improvement. Patient-acceptable symptomatic state (PASS) and substantial clinical benefit (SCB) ultimately correlate with whether patients are happy or willing to undergo an intervention again. These measures should not be reported in terms of group means, where outliers could distort the score. These measures should be reported in terms of the proportion of individual patients whose scores cross the meaningful thresholds of satisfaction.

Tools to Improve Scientific Research

The mission of Arthroscopy is to provide authoritative, current, peer-reviewed clinical and basic science information regarding arthroscopic and related surgery. In addition, with a goal of improving the quality of the scientific research published in our journal and others, we develop and publish research pearls, statistical guides, article checklists and templates, and related tools. In sum, this effort allows our cadre of editors, reviewers, authors, and readers to strive to improve in our ability to create and critically analyze medical literature of the greatest merit. Our ultimate ambition is to publish more perfect articles with conclusions on which readers can rely.