Interpreting statistics in medical literature: a vade mecum for surgeons

Errors and complications in clinical periodontal practice due to methodologic bias and bad interpretation of the evidence

Different types of errors and complications may arise during and after the execution of periodontal or implant-related procedures. Some of the most relevant, although also controversial, and less commented, causative agents of errors and complications are methodological biases and bad interpretation of the evidence. Proper assessment of the literature requires of solid clinical knowledge combined with a systematic approach built on the recognition of common methodological biases and the avoidance of interpretive errors to critically retrieve, dissect, and judiciously apply available information for the promotion of periodontal and peri-implant health. This review addresses common types of methodological bias and interpretive errors that can alter the reader's perceptions on the real effect and potential ramifications of the reported outcomes of a given therapeutic approach due to bad interpretation of the available evidence: (1) types of methodological biases; (2) spin and interpretive bias; (3) interpretation pitfalls when assessing the evidence (4) choice of relevant endpoints to answer the question(s) of interest; and (5) balance between statistical significance and clinical relevance.

An analysis of statistical terminology applied in emergency medicine literature methods

INTRODUCTION

Given the importance of understanding methodical reporting and statistical terminology in ensuring evidence-based decision-making, physicians should possess statistical literacy. The purpose of this study was to distinguish statistical terminology commonly used in emergency medicine methods and describe changes in statistical methods from 2011 to 2021.

METHODS

The research team chose four journals in emergency medicine widely read and clinically relevant: Academic Emergency Medicine, American Journal of Emergency Medicine, Annals of Emergency Medicine, and Journal of Emergency Medicine. A total of 400 research articles were included. The team compared overall frequencies in statistical word counts as well as differences by year (2021 vs 2011).

RESULTS

Included words from 2011 totaled 31,002 compared to 38,272 in 2021. Unique words for 2011 included 3801 compared to 4291 in 2021. The most common statistical words found in both years were model(s), difference(s), and regression(s). The largest increases in usage included the following words: noninferior(ity), NPV, Fixed, AUC, Mixed, Shapiro, and Wilk. Compared to 2011, results showed a 10% decrease in "p" for 2021. While the terms "Confidence" (2%) and "Intervals" (6%) decreased from 2011 to 2021, there was a 25% increase in "CI(s)".

CONCLUSION

By understanding common statistical terms and trends over time, educational efforts can be targeted to consumers of EM literature. Additionally, this work provides evidence suggesting an overall improvement in processes in statistical methodology, enhancing the quality of research outputs and ultimately allowing better clinical decision-making.

Advanced Statistics: Multiple Logistic Regression, Cox Proportional Hazards, and Propensity Scores

Background: Randomized controlled trials (RCTs) are generally regarded as the gold standard for demonstrating causality because they effectively mitigate bias from both known and unknown confounders. However, conducting an RCT is not always feasible because of logistical and ethical considerations. This is especially true when evaluating surgical interventions, and non-randomized study designs must be utilized instead. Methods: Statistical methods that adjust for baseline differences in non-randomized studies were reviewed. Results: The three methods used most commonly to adjust for confounding factors are multiple logistic regression, Cox proportional hazard, and propensity scoring. Multiple logistic regression (MLR) is implemented to analyze the influence of categorical and/or continuous variables on a single dichotomous outcome. The model controls for multiple covariates while also quantifying the magnitude of each covariate's influence on the outcome. Selecting which variables to include in a model should be the most important consideration, and authors must report how and why variables were chosen. Cox proportional hazards modeling is conceptually similar to logistic regression and is used when analyzing survival data. When applied to survival curves, Cox proportional hazards can adjust for baseline group differences and provide a hazard ratio to quantify the effect that any single factor contributes to the survival curve. Propensity scores (PS) range from zero to one and are defined as the probability of receiving an intervention based on observed baseline characteristics. Propensity score matching (PSM) is especially useful when the outcome of interest is a rare event. Treated and untreated subjects with similar propensity scores are paired, forming balanced samples for further analysis. Conclusions: The method by which to address confounding should be selected according to the data format and sample size. Reporting of methods should provide justification for selected covariates, confirmation that data did not violate model assumptions, and measures of model performance.

Comparison of the Kansas City Cardiomyopathy Questionnaire and Minnesota Living With Heart Failure Questionnaire in Predicting Heart Failure Outcomes

Patient-reported outcome measures (PROMs) are relevant independent outcomes in heart failure (HF) care and are predictive of subsequent hospitalization and death in HF. The Kansas City Cardiomyopathy Questionnaire (KCCQ) and the Minnesota Living with Heart Failure Questionnaire (MLHFQ) are the 2 most widely adopted PROMs specific to HF. We compared their prognostic abilities in a prospective cohort of HF patients. A prospective cohort of subjects from a single-center registry was analyzed with regard to baseline KCCQ and MLHFQ scores and the outcomes of death, transplant, or left ventricular assist device implantation and hospitalization. A total of 516 subjects with reduced left ventricular ejection fraction (HFrEF) and 151 subjects with preserved left ventricular ejection fraction (HFpEF) were included. Discrimination was assessed using c-statistics based on time-to-event analyses and receiver-operator curves. The additive contribution of MLHFQ was assessed through the change in c-statistic, incremental discrimination index, and category-free net reclassification index. Overall, KCCQ was superior to MLHFQ for predicting death/transplant/ventricular assist device (c-statistic 0.702 [0.666 to 0.738] and 0.658 [0.621 to 0.695] respectively, p value for difference <0.001) and hospitalization (c-statistic 0.640 [0.613 to 0.666] and 0.624 [0.597 to 0.651], respectively, p value for difference 0.022). However, this difference was statistically nonsignificant in the HFpEF group alone. When analyzing the additional prognostic information afforded by adding MLHFQ to KCCQ in the overall, HFrEF, and HFpEF groups there was no significant improvement, although adding KCCQ to MLHFQ did significantly improve risk stratification. Scoring based upon the abbreviated KCCQ-12 did not reduce the prognostic accuracy of KCCQ. In conclusion, KCCQ is more prognostic of death/transplant/left ventricular assist device and hospitalization than MLHFQ in a combined cohort of patients with HFrEF and HFpEF, although the effect in HFpEF was less pronounced. KCCQ should be the preferred PROM for patients with HF if prognostication is a desired goal of using the PROMs.

Publication rates of full-text journal articles converted from abstracts presented during the 22(nd) Turkish National Urology Congress

OBJECTIVE

To analyze the publication rates of full-text journal articles converted from the abstracts presented in the 22(nd) Turkish National Urology Congress in 2012.

MATERIAL AND METHODS

A total of 576 abstracts accepted for presentation at the 22(nd) Turkish National Urology Association Meeting were identified from the published abstract book. The abstracts were categorized into subsections such as endourology and pediatric urology. The subsequent publication rate for the studies was evaluated by scanning PubMed Medline. Abstracts published before the proceedings were excluded from the study.

RESULTS

The abstracts were categorized as being presented orally (n=155), by poster (n=421), or by video (n=78). Of the 28 (18.3%) of 155 oral and 34 (8.15%) of 421 poster presentations, were subsequently published in several journals until March 2015. The publication rates of the abstracts based on urology subsections were as follows: neurology (25%), andrology (18.6%), endourology (17.2%), urolithiasis (15.3%), general urology (12.5%), infectious diseases (7.14%), pediatric urology (6.25%), uro-gynecology (6.06%), reconstructive urology (5.8%), and urooncology (3.8%). The average time to publication was 11.77 (0-33) months.

CONCLUSION

This is the first study assessing the publication rates of abstracts presented at a Turkish National Urology Congress. It reveals that more qualified randomized studies need to be done to improve the rate of publication.

How to review a surgical paper: a guide for junior referees

Reviewing a surgical manuscript is not an easy task, and there is no formal training available for young referees in the early stage of their careers. Accepting a peer review assignment represents a personal honor for the invited referee and a fundamental ethical responsibility towards the scientific community. Designated reviewers must be accomplished and knowledgeable in the area of the respective topic of investigation. More importantly, they must be aware and cognizant about the cardinal ethical responsibility and stewardship for ensuring the preservation of scientific knowledge of unbiased and unquestionable accuracy in the published literature. Accepting a review assignment should never be taken lightly or considered a simple task, regardless of the reviewer's level of seniority and expertise. Indeed, there are multiple challenges, difficulties, and 'hidden dangers' that jeopardize the completion of a high-quality review, particularly in the hands of less experienced or novice reviewers. The present article was designed to provide a brief, concise, and practical guide on how to review manuscripts for the 'junior referee' in the field of surgery.

Is dental caries experience increased in HIV-infected children and adolescents? A meta-analysis

OBJECTIVE

To undertake a systematic review to assess if HIV-infected children and adolescents have an increased dental caries experience.

METHODS

A search of MEDLINE, BIREME, EMBASE, GOOGLE SCHOLAR, SIGLE (Grey Literature) and reference lists of included studies was carried out. To be eligible the studies had to present HIV-infected and non-infected children/adolescents between 0-18 years old. To assess the methodological quality, the studies were categorized in scores from 'A' to 'C'. To perform a meta-analysis a random effect model was used with 95% confidence intervals and two distinct sub-group analyses were carried out in terms of caries progression: data for cavitated and non-cavitated lesions (sub-group 1) and data only for cavitated lesions (sub-group 2).

RESULTS

Five studies fulfilled the selection criteria. Four studies (two ranked A and two B in the quality assessment) revealed higher caries scores in primary teeth in the HIV-infected patients with mean dmft/dmfs scores of 3.8-4.1/7.8-11.0 compared to the control group 1.5-2.4/3.4-5.1. No differences in caries index were found for permanent dentition. The meta-analysis excluded caries data of permanent teeth and showed a significant association between caries experience in primary dentition and HIV infection considering cavitated and non-cavitated lesions (OR = 2.33, 95% CI = 1.48-3.68) or only cavitated lesions (OR = 2.98, 95% CI = 1.59-5.59).

CONCLUSION

Evidence exists that suggests HIV-infected children/adolescents have an increased caries experience in primary dentition.

Comparable performance of the Kansas City Cardiomyopathy Questionnaire in patients with heart failure with preserved and reduced ejection fraction

BACKGROUND

Despite the growing epidemic of heart failure with preserved ejection fraction (HFpEF), no valid measure of patients' health status (symptoms, function, and quality of life) exists. We evaluated the Kansas City Cardiomyopathy Questionnaire (KCCQ), a validated measure of HF with reduced EF, in patients with HFpEF.

METHODS AND RESULTS

Using a prospective HF registry, we dichotomized patients into HF with reduced EF (EF≤ 40) and HFpEF (EF≥50). The associations between New York Heart Association class, a commonly used criterion standard, and KCCQ Overall Summary and Total Symptom domains were evaluated using Spearman correlations and 2-way ANOVA with differences between patients with HF with reduced EF and HFpEF tested with interaction terms. Predictive validity of the KCCQ Overall Summary scores was assessed with Kaplan-Meier curves for death and all-cause hospitalization. Covariate adjustment was made using Cox proportional hazards models. Internal reliability was assessed with Cronbach's α. Among 849 patients, 200 (24%) had HFpEF. KCCQ summary scores were strongly associated with New York Heart Association class in both patients with HFpEF (r=-0.62; P<0.001) and HF with reduced EF (r=-0.55; P=0.27 for interaction). One-year event-free rates by KCCQ category among patients with HFpEF were 0 to 25=13.8%, 26 to 50=59.1%, 51 to 75=73.8%, and 76 to 100=77.8% (log rank P<0.001), with no significant interaction by EF (P=0.37). The KCCQ domains demonstrated high internal consistency among patients with HFpEF (Cronbach's α=0.96 for overall summary and ≥0.69 in all subdomains).

CONCLUSIONS

Among patients with HFpEF, the KCCQ seems to be a valid and reliable measure of health status and offers excellent prognostic ability. Future studies should extend and replicate our findings, including the establishment of its responsiveness to clinical change.

Bayesian statistics in oncology: a guide for the clinical investigator

The rise of evidence-based medicine as well as important progress in statistical methods and computational power have led to a second birth of the >200-year-old Bayesian framework. The use of Bayesian techniques, in particular in the design and interpretation of clinical trials, offers several substantial advantages over the classical statistical approach. First, in contrast to classical statistics, Bayesian analysis allows a direct statement regarding the probability that a treatment was beneficial. Second, Bayesian statistics allow the researcher to incorporate any prior information in the analysis of the experimental results. Third, Bayesian methods can efficiently handle complex statistical models, which are suited for advanced clinical trial designs. Finally, Bayesian statistics encourage a thorough consideration and presentation of the assumptions underlying an analysis, which enables the reader to fully appraise the authors' conclusions. Both Bayesian and classical statistics have their respective strengths and limitations and should be viewed as being complementary to each other; we do not attempt to make a head-to-head comparison, as this is beyond the scope of the present review. Rather, the objective of the present article is to provide a nonmathematical, reader-friendly overview of the current practice of Bayesian statistics coupled with numerous intuitive examples from the field of oncology. It is hoped that this educational review will be a useful resource to the oncologist and result in a better understanding of the scope, strengths, and limitations of the Bayesian approach.

Critical appraisal of meta-analyses: an introductory guide for the practicing surgeon

Meta-analyses are an essential tool of clinical research. Meta-analyses of individual randomized controlled trials frequently constitute the highest possible level of scientific evidence for a given research question and allow surgeons to rapidly gain a comprehensive understanding of an important clinical issue. Moreover, meta-analyses often serve as cornerstones for evidence-based surgery, treatment guidelines, and knowledge transfer. Given the importance of meta-analyses to the medical (and surgical) knowledge base, it is of cardinal importance that surgeons have a basic grasp of the principles that guide a high-quality meta-analysis, and be able to weigh objectively the advantages and potential pitfalls of this clinical research tool. Unfortunately, surgeons are often ill-prepared to successfully conduct, critically appraise, and correctly interpret meta-analyses. The objective of this educational review is to provide surgeons with a brief introductory overview of the knowledge and skills required for understanding and critically appraising surgical meta-analyses as well as assessing their implications for their own surgical practice.

Caveats in the interpretation of the surgical literature

The ability to appraise research literature critically is an invaluable tool in the era of evidence-based medicine. The objective of this educational article is to highlight certain caveats that are important to those who seek to interpret the surgical literature. These caveats are illustrated with intuitive examples drawn from the field of surgery. It is hoped that the article will help surgeons to recognize the common pitfalls encountered when interpreting scientific reports. This is important if research findings are to be transferred appropriately into the realm of clinical practice.

What happens to the abstracts presented at the Societè Internationale d'Urologie meeting?

OBJECTIVES

To estimate the acceptance rate on peer-reviewed journals, describe the time-course of subsequent full publication, and identify those with characteristics associated with publication from the abstracts presented at the Societè Internationale d'Urologie (SIU) meeting.

METHODS

All abstracts accepted for presentation at the annual 2002 and 2004 SIU meetings were identified from the published supplements in the BJU International. The subsequent publication rate for the corresponding studies by scanning Medline was evaluated.

RESULTS

Overall, 1877 abstracts were presented, 415 of which (22.1%) were followed by publication in peer-reviewed journals. The mean time to publication was 13 months (range, 1 to 45 months). A total of 60.6% of the published articles were in print within 1 year of presentation "In most of the cases the reports were published in The Journal of Urology (76 reports) and in Urology (49 reports). Abstracts on kidney cancer, urinary diversion, incontinence, and urolithiasis had the highest publication rates (43.4%, 36.6%, 33.3%, and 29.2%, respectively). Studies from North America and from Oceania had the highest publication rate (26.8% and 26.6%, respectively).

CONCLUSIONS

One fifth of the abstracts presented at the SIU meetings are ultimately published in indexed journals. Delegates attending these meetings should be aware of this issue when taking into account findings from meeting reports for their clinical practice. Factors influencing their publication are the abstract geographical area of origin, the study subject, and the research type. The Journal of Urology and Urology represented target journals for the publication more than other ones.

International integrated database for the evaluation of severe sepsis and drotrecogin alfa (activated) therapy: analysis of efficacy and safety data in a large surgical cohort

BACKGROUND

The International Integrated Database for the Evaluation of Severe Sepsis and Drotrecogin alfa (activated) Therapy includes an extensive cohort of surgical patients (1659/4459; 37%). This database broadens the experience reported on a comparatively small set of surgical patients from the pivotal Protein C Worldwide Evaluation in Severe Sepsis trial to examine issues of safety and efficacy in a much larger cohort.

METHODS

We conducted a retrospective analysis of prospectively defined outcomes from 5 integrated clinical studies of severe sepsis. Multivariable analyses incorporated propensity scores, treatment, and significant baseline risk factors as independent variables in logistic regression models for 2 outcomes: serious adverse events that were observed during infusion and 28-day, all-cause mortality rates. Adjusted odds ratios were calculated for clinically important strata. Multiple subcategories of serious bleeding-event rates are presented.

RESULTS

Although surgical patients who were treated with drotrecogin alfa [activated] (DrotAA) experienced a greater proportion of serious bleeding events during the infusion period, most of the patients were treated without fatal consequence. A 10.7% absolute all cause mortality risk reduction (adjusted odds ratio, 0.66; 95% CI, 0.45-0.97) was observed for DrotAA-treated, high-risk (Acute Physiology and Chronic Health Evaluation II, >/= 25) surgical patients. We could not demonstrate a survival benefit in DrotAA-treated, low-risk (Acute Physiology and Chronic Health Evaluation II, <25) surgical patients. When surgical patients were stratified by number of organ dysfunctions, absolute risk reductions were observed in both categories: multiorgan (4.3%) and single (4.5%).

CONCLUSION

International Integrated Database for the Evaluation of Severe Sepsis and Drotrecogin alfa (activated) Therapy analyses affirmed the favorable benefit/risk profile of DrotAA for surgical patients. The serious adverse event rate that was experienced by surgical patients during the study drug infusion period was 7.5% in the DrotAA-treated group versus 6.3% in the placebo-treated group (odds ratio, 1.41; 95% CI, 0.89-2.25). The clinical benefit of DrotAA therapy paralleled baseline risk of death and substantiated findings from the Protein C Worldwide Evaluation in Severe Sepsis study. Future analyses are needed to evaluate the special relationships among sepsis severity, bleeding management, and the postoperative timing of DrotAA administration.

Factors associated with the full publication of studies presented in abstract form at the annual meeting of the American Urological Association

PURPOSE

Many abstracts presented at scientific meetings never come to full text publication, which is a prerequisite for the critical appraisal of a given study for its validity, impact and generalizability. We determined factors associated with the publication of abstracts presented at the American Urological Association national meeting.

MATERIALS AND METHODS

All abstracts addressing clinical research accepted for presentation at the 2002 and 2003 meetings of the American Urological Association were reviewed. A comprehensive MEDLINE search was performed for evidence of publication in full manuscript form. Data abstraction and literature searches were done between June 15 and August 30, 2005. Univariate and multivariate analyses were performed to determine the association between abstract characteristics and time to publication.

RESULTS

Of the 1,683 abstracts reviewed 740 (44.0%) were published within a median followup of 27.8 months (range 25.9 to 39.7). Time to publication was associated with abstract origin in the United States and the reporting of statistical testing (HR 1.2, 95% CI 1.0-1.4, p=0.040 and HR 1.2, 95% CI 1.1-1.4, p=0.010, respectively). Other variables, such as presentation type, study design, clinical question type and negative outcome, were not predictive.

CONCLUSIONS

Nonpublication of research findings is a problematic issue that affects more than half of studies 2 years after presentation at the American Urological Association national meeting. Abstracts from the United States and those providing statistical testing were more likely to be published in full text form. Further efforts are warranted to identify and eliminate factors that hinder publication of research to bring it to the scrutiny of a broad audience of urologists.

Non-Inferiority Trials in Surgical Oncology

The classical randomized controlled clinical trial is designed to prove superiority of an investigational therapy over an established therapy or placebo (here referred to as "superiority trial"). Although the randomized controlled superiority trial has its well-grounded role, clinical trials of non-inferiority are equally important in the advance of medical science. Non-inferiority trials test whether a new intervention is as good as a standard treatment with respect to curing the illness (e.g., overall survival) while offering other benefits over the standard therapy, such as lower toxicity, better side-effect profile, improved ease of administration, or reduced costs. The evaluation of non-inferiority is critical in many settings. In surgical oncology, for instance, treatments often combine advantages (e.g., survival benefit) with disadvantages (e.g., high post-operative morbidity due to extensive surgery, considerable toxic effects of an aggressive chemotherapy regimen). The various aspects of different therapeutic strategies may make a treatment decision difficult, requiring a non-inferiority trial to quantify risks and benefits. However, despite their great importance in clinical cancer research, the concept, design, and objectives of non-inferiority trials remain poorly understood in the surgical community. The goal of this review is to discuss the principles, strengths, and challenges of non-inferiority trials and introduce this highly relevant topic to the surgical reader, using examples from the field of surgical oncology.

International integrated database for the evaluation of severe sepsis and drotrecogin alfa (activated) therapy: analysis of efficacy and safety data in a large surgical cohort

BACKGROUND

The International Integrated Database for the Evaluation of Severe Sepsis and Drotrecogin alfa (activated) Therapy includes an extensive cohort of surgical patients (1659/4459; 37%). This database broadens the experience reported on a comparatively small set of surgical patients from the pivotal Protein C Worldwide Evaluation in Severe Sepsis trial to examine issues of safety and efficacy in a much larger cohort.

METHODS

We conducted a retrospective analysis of prospectively defined outcomes from 5 integrated clinical studies of severe sepsis. Multivariable analyses incorporated propensity scores, treatment, and significant baseline risk factors as independent variables in logistic regression models for 2 outcomes: serious adverse events that were observed during infusion and 28-day, all-cause mortality rates. Adjusted odds ratios were calculated for clinically important strata. Multiple subcategories of serious bleeding-event rates are presented.

RESULTS

Although surgical patients who were treated with drotrecogin alfa [activated] (DrotAA) experienced a greater proportion of serious bleeding events during the infusion period, most of the patients were treated without fatal consequence. A 10.7% absolute all cause mortality risk reduction (adjusted odds ratio, 0.66; 95% CI, 0.45-0.97) was observed for DrotAA-treated, high-risk (Acute Physiology and Chronic Health Evaluation II, >or=25) surgical patients. We could not demonstrate a survival benefit in DrotAA-treated, low-risk (Acute Physiology and Chronic Health Evaluation II, <25) surgical patients. When surgical patients were stratified by number of organ dysfunctions, absolute risk reductions were observed in both categories: multiorgan (4.3%) and single (4.5%).

CONCLUSION

International Integrated Database for the Evaluation of Severe Sepsis and Drotrecogin alfa (activated) Therapy analyses affirmed the favorable benefit/risk profile of DrotAA for surgical patients. The serious adverse event rate that was experienced by surgical patients during the study drug infusion period was 7.5% in the DrotAA-treated group versus 6.3% in the placebo-treated group (odds ratio, 1.41; 95% CI, 0.89-2.25). The clinical benefit of DrotAA therapy paralleled baseline risk of death and substantiated findings from the Protein C Worldwide Evaluation in Severe Sepsis study. Future analyses are needed to evaluate the special relationships among sepsis severity, bleeding management, and the postoperative timing of DrotAA administration.

Interpreting Statistics in the Urological Literature

PURPOSE

Knowledge of statistical terminology and the ability to critically interpret research findings are critical skills in the practice of evidence based medicine.

MATERIALS AND METHODS

We provide a series of nontechnical explanations of basic statistical concepts commonly encountered in the urological literature. In addition, we provide examples of common statistical pitfalls to increase awareness of limitations to consider when applying research findings to practice.

RESULTS

Statistical goals encountered in the urological literature can be broadly categorized as summarizing outcome variables, comparing 2 or more groups, measuring association among variables or predicting 1 variable from another. Errors frequently include the use of an inappropriate test for the data type of interest or using statistical testing in a manner that increases the likelihood of false-positive results. Such errors pose a threat to the validity of research findings and they may undermine study conclusions.

CONCLUSIONS

Editors and reviewers alike should strive for high standards of statistical analysis and reporting, and promote the publication of high quality evidence in the urological literature. The understanding of basic statistical concepts and the principles of the hypothesis testing framework is essential to the critical appraisal process and, therefore, important to all urologists. Statistical literacy should be fostered through educational materials and courses in the urological community.

Propensity scores and the surgeon

BACKGROUND

Evidence-based surgery has been established as a cornerstone of good clinical practice, promising to improve the treatment of patients and the quality of surgical education. However, evidence-based surgery requires dedicated clinicians trained to perform methodologically sound clinical investigations. Statistical knowledge is therefore invaluable. Surgical studies often cannot be randomized. Propensity scores offer a powerful alternative to multivariable analysis in the assessment of observational, non-randomized surgical studies. Unfortunately, many surgeons are unaware of this important analytical approach that has gained increasing stature in medical research. Thus, propensity score analyses are not used often in surgical studies.

OBJECTIVE

The purpose of this paper is to provide a comprehensive overview of propensity score analysis, allowing the surgeon to understand the role, advantages and limitations of propensity scores, boosting their development in surgical investigations.

Admission serum lactate levels do not predict mortality in the acutely injured patient

INTRODUCTION

The conventional view that admission lactate levels predict outcome in trauma patients stems from simple comparisons of mean blood levels between groups and small sample sizes. To better address this question, we performed more rigorous statistical analyses of lactate in a larger patient sample.

METHODS

We prospectively collected data on admission lactate and outcomes in 5,995 patients admitted to an urban, university-based trauma center. The ability of admission lactate to predict mortality was assessed by logistic regression, calculation of positive predictive values (PPV), and measurement of areas under receiver operating characteristic (ROC) curves.

RESULTS

Differences between survivors and nonsurvivors in means of most proposed prognosticators was again demonstrated. However, the large overlap in these variables between survivors and nonsurvivors prevented clinically useful predictions. The overall PPV of elevated lactate was only 5.4%. Even in severely injured patients (Injury Severity Score >20; mortality 23%), elevated admission lactate level was a poor predictor of outcome. ROC analyses found no useful sensitivity threshold overall or after stratification by age, sex, Glasgow Coma Scale score, revised trauma score, or mechanism of injury.

CONCLUSIONS

This large retrospective examination of admission lactate levels failed to show useful predictive accuracy for hospital death. Serum lactate levels need not be obtained routinely but can be reserved for patients who will be admitted to the intensive care unit and/or require an emergency operation.

Surgical Outcomes Research Based on Administrative Data: Inferior or Complementary to Prospective Randomized Clinical Trials?

The importance of surgical research has gained new prominence over the past decades as the relevance of well designed and well conducted studies has become increasingly evident. There are two basic but diametrically different methods of conducting research: the prospective randomized clinical trial and the retrospective surgical outcomes study based on administrative data. Administrative databases contain data that were initially collected for purposes other than scientific research. Whereas the prospective randomized clinical trial is familiar to most surgeons, surgical outcomes research based on administrative data constitutes a genre of investigation that is often unfamiliar to and even disparaged by the surgical community. In the present article, the strengths and weaknesses of both prospective randomized clinical trials and retrospective surgical outcomes research are discussed. Specifically, the advantages and limitations of investigations based on large administrative databases are outlined. Because both study designs play an important role in surgical research, carefully designed and implemented surgical outcomes research based on administrative data should be viewed as being complementary and not inferior to prospective randomized clinical trials.

Detecting pheochromocytoma: defining the most sensitive test

OBJECTIVE

To define the most sensitive biochemical test to establish the diagnosis of pheochromocytoma and also to assess the potential role of iodine 131-labeled metaiodobenzylguanidine scintigraphy (I-MIBG) in the diagnosis of this tumor.

SUMMARY BACKGROUND DATA

Pheochromocytoma is a rare, catecholamine-producing tumor with preferential localization in the adrenal gland. Despite its importance, the most sensitive test to establish the diagnosis remains to be defined.

METHODS

Prospective data collection was done on patients with pheochromocytoma treated at the Duke University Medical Center and the Durham Veterans Affairs Medical Center, Durham, NC. All urinary, plasma, and platelet analyses were highly standardized and supervised by one investigator (J.M.F.). I-MIBG scans were independently reviewed by 2 nuclear medicine physicians.

RESULTS

A total of 152 patients (55.3% female) were enrolled in the present analysis. Patients were predominantly white (73.7%). Spells (defined as profuse sweating, tachycardia, and headache) and hypertension at diagnosis were present in 51.4% and 66.6%, respectively. Bilateral disease was found in 12.5%, malignant pheochromocytoma in 29.6%, and hereditary forms in 23.0%. The most sensitive tests were total urinary normetanephrine (96.9%), platelet norepinephrine (93.8%), and I-MIBG scintigraphy (83.7%). In combination with I-MIBG scintigraphy, platelet norepinephrine had a sensitivity of 100%, plasma norepinephrine/MIBG of 97.1%, total urine normetanephrine/MIBG of 96.6%, and urine norepinephrine/MIBG of 95.3%.

CONCLUSIONS

The tests of choice to establish the diagnosis of pheochromocytoma are urinary normetanephrine and platelet norepinephrine. A combination of I-MIBG scintigraphy and diagnostic tests in urine, blood, or platelets does further improve the sensitivity. We thus advocate performing an MIBG scan if the diagnosis of pheochromocytoma is clinically suspected and catecholamine measurements are within the normal range.

Statistical methods in the surgical literature

BACKGROUND

It is important that clinicians understand statistical methods to incorporate statistics into their own research and to correctly translate published literature into improved patient care. The purpose of this study was to identify frequency and appropriate use of statistical methods in clinical surgical publications during the past 18 years.

STUDY DESIGN

The study included randomly selected issues from odd-numbered years of Annals of Surgery (Annals) and Archives of Surgery (Archives) between 1985 and 2003, and issues in 2003 from Journal of the American College of Surgeons (JACS), Journal of Surgical Research (JSR), and Surgery. We identified all statistical procedures reported in each article, examined correctness of methods, and reported trends in publication of statistical methods over time.

RESULTS

The proportion of publications incorporating statistics has increased over time. Declining trends were seen in the proportion of articles with no statistics (p < 0.0001). Approximately 35% of articles in 1985 did not use statistics compared with < 10% in 2003. Nonparametric tests increased (p < 0.0001) during the study period. In Archives of Surgery, nonparametric tests increased from 0% in 1985 to 33% in 2003, and in Annals of Surgery, from 12% in 1985 to 49% in 2003. Twenty-seven percent of studies included incorrect selection or reporting of statistical methods.

CONCLUSIONS

Overall, the statistical complexity of research in clinical surgery journals is increasing. It is important that clinicians reading this literature have sufficient knowledge of statistical methods to facilitate interpretation of increasingly sophisticated statistical analyses.

Evidence-based medicine in pathology: an introduction

Evidence-based medicine (EBM) is a school of thought that has spread rapidly through medicine in the past 2 decades and is eliciting an increasing interest in Anatomic Pathology and Laboratory Medicine. It has been defined as "the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients." The environmental factors that created a need for EBM and basic concepts of this discipline are reviewed. Methods for the accrual and critical appraisal of the validity of available evidence and its impact, applicability and usefulness in pathology practice are discussed. Basic concepts of bayesian data analysis with an emphasis on concepts such as prior and posterior probability and the use of "holdout" or "test" data are introduced. The future of EBM in pathology is discussed and potential applications of these concepts to pathology practice and research are proposed.

Sample Size Matters: A Guide for Surgeons

Considerations of sample size computations in the medical literature have gained increasing importance over the past decade and are now often mandatory for scientific grant proposals, protocols, and publications. However, many surgeons are ill-prepared to understand the parameters on which the appropriate sample size is based. The present article has several objectives: first, to review the need for sample size considerations; second, to explain the ingredients necessary for sample size computations in simple, nonmathematic language; third, to provide options for reducing the sample size if it seems impracticably large; and fourth, to help avoid some of the more common mistakes encountered when computing sample sizes.