The task of a statistical referee

Peer review of clinical and translational research manuscripts: Perspectives from statistical collaborators

Research articles in the clinical and translational science literature commonly use quantitative data to inform evaluation of interventions, learn about the etiology of disease, or develop methods for diagnostic testing or risk prediction of future events. The peer review process must evaluate the methodology used therein, including use of quantitative statistical methods. In this manuscript, we provide guidance for peer reviewers tasked with assessing quantitative methodology, intended to complement guidelines and recommendations that exist for manuscript authors. We describe components of clinical and translational science research manuscripts that require assessment including study design and hypothesis evaluation, sampling and data acquisition, interventions (for studies that include an intervention), measurement of data, statistical analysis methods, presentation of the study results, and interpretation of the study results. For each component, we describe what reviewers should look for and assess; how reviewers should provide helpful comments for fixable errors or omissions; and how reviewers should communicate uncorrectable and irreparable errors. We then discuss the critical concepts of transparency and acceptance/revision guidelines when communicating with responsible journal editors.

Glomerular hyperfiltration and hypertrophy: an evaluation of maximum values in pathological indicators to discriminate "diseased" from "normal"

The success of sodium-glucose cotransporter 2 inhibitors and bariatric surgery in patients with chronic kidney disease has highlighted the importance of glomerular hyperfiltration and hypertrophy in the progression of kidney disease. Sustained glomerular hyperfiltration and hypertrophy can lead to glomerular injury and progressive kidney damage. This article explores the relationship between obesity and chronic kidney disease, focusing on the roles of glomerular hyperfiltration and hypertrophy as hallmarks of obesity-related kidney disease. The pathological mechanisms underlying this association include adipose tissue inflammation, dyslipidemia, insulin resistance, chronic systemic inflammation, oxidative stress, and overactivation of the sympathetic nervous system, as well as the renin-angiotensin aldosterone system. This article explains how glomerular hyperfiltration results from increased renal blood flow and intraglomerular hypertension, inducing mechanical stress on the filtration barrier and post-filtration structures. Injured glomeruli increase in size before sclerosing and collapsing. Therefore, using extreme values, such as the maximal glomerular diameter, could improve the understanding of the data distribution and allow for better kidney failure predictions. This review provides important insights into the mechanisms underlying glomerular hyperfiltration and hypertrophy and highlights the need for further research using glomerular size, including maximum glomerular profile, calculated using needle biopsy specimens.

Basics, common errors and essentials of statistical tools and techniques in anesthesiology research

The statistical portion is a vital component of any research study. The research methodology and the application of statistical tools and techniques have evolved over the years and have significantly helped the research activities throughout the globe. The results and inferences are not accurately possible without proper validation with various statistical tools and tests. The evidencebased anesthesia research and practice has to incorporate statistical tools in the methodology right from the planning stage of the study itself. Though the medical fraternity is well acquainted with the significance of statistics in research, there is a lack of in-depth knowledge about the various statistical concepts and principles among majority of the researchers. The clinical impact and consequences can be serious as the incorrect analysis, conclusions, and false results may construct an artificial platform on which future research activities are replicated. The present tutorial is an attempt to make anesthesiologists aware of the various aspects of statistical methods used in evidence-based research and also to highlight the common areas where maximum number of statistical errors are committed so as to adopt better statistical practices.

How to spot a statistical problem: advice for a non-statistical reviewer

Statistical analyses presented in general medical journals are becoming increasingly sophisticated. BMC Medicine relies on subject reviewers to indicate when a statistical review is required. We consider this policy and provide guidance on when to recommend a manuscript for statistical evaluation. Indicators for statistical review include insufficient detail in methods or results, some common statistical issues and interpretation not based on the presented evidence. Reviewers are required to ensure that the manuscript is methodologically sound and clearly written. Within that context, they are expected to provide constructive feedback and opinion on the statistical design, analysis, presentation and interpretation. If reviewers lack the appropriate background to positively confirm the appropriateness of any of the manuscript's statistical aspects, they are encouraged to recommend it for expert statistical review.

Scientific and statistical reviews of manuscripts submitted to Nursing Research: Comparison of completeness, quality, and usefulness

Sound statistical analysis is fundamental to high-quality reporting of quantitative studies. Peer review are also important in determining whether statistical procedures in submitted manuscripts are appropriate and effectively reported. However, the quality of statistical reviews of manuscripts submitted to nursing journals has not been previously evaluated. The purpose of this project was to compare the content and quality of statistical and scientific (general) reviews of manuscripts submitted to Nursing Research. The General Assessment of Reviews of Nursing Research (GARNR) and the Statistical Assessment of Reviews of Nursing Research (SARNR) were used to evaluate 105 reviews of 35 manuscripts assigned to both scientific and statistical review between August 2007 and February 2009. Scientific reviews were rated as more comprehensive, but most did not evaluate statistical aspects of a manuscript. Statistical reviews were more likely to identify fatal flaws, were generally rated higher in overall usefulness to the editor in making a decision on whether to publish, and were rated as more useful to authors for improving a manuscript. Statistical and scientific reviews are complementary and both are necessary. More thorough integration of substantive and methodological content in reviews of quantitative studies has potential for even greater leverage of effort and improved quality of nursing research reports.

Statistical methods in the surgical literature

BACKGROUND

It is important that clinicians understand statistical methods to incorporate statistics into their own research and to correctly translate published literature into improved patient care. The purpose of this study was to identify frequency and appropriate use of statistical methods in clinical surgical publications during the past 18 years.

STUDY DESIGN

The study included randomly selected issues from odd-numbered years of Annals of Surgery (Annals) and Archives of Surgery (Archives) between 1985 and 2003, and issues in 2003 from Journal of the American College of Surgeons (JACS), Journal of Surgical Research (JSR), and Surgery. We identified all statistical procedures reported in each article, examined correctness of methods, and reported trends in publication of statistical methods over time.

RESULTS

The proportion of publications incorporating statistics has increased over time. Declining trends were seen in the proportion of articles with no statistics (p < 0.0001). Approximately 35% of articles in 1985 did not use statistics compared with < 10% in 2003. Nonparametric tests increased (p < 0.0001) during the study period. In Archives of Surgery, nonparametric tests increased from 0% in 1985 to 33% in 2003, and in Annals of Surgery, from 12% in 1985 to 49% in 2003. Twenty-seven percent of studies included incorrect selection or reporting of statistical methods.

CONCLUSIONS

Overall, the statistical complexity of research in clinical surgery journals is increasing. It is important that clinicians reading this literature have sufficient knowledge of statistical methods to facilitate interpretation of increasingly sophisticated statistical analyses.

Use of statistical procedures in Brazilian and international dental journals

A descriptive survey was performed in order to assess the statistical content and quality of Brazilian and international dental journals, and compare their evolution throughout the last decades. The authors identified the reporting and accuracy of statistical techniques in 1000 papers published from 1970 to 2000 in seven dental journals: three Brazilian (Brazilian Dental Journal, Revista de Odontologia da Universidade de Sao Paulo and Revista de Odontologia da UNESP) and four international journals (Journal of the American Dental Association, Journal of Dental Research, Caries Research and Journal of Periodontology). Papers were divided into two time periods: from 1970 to 1989, and from 1990 to 2000. A slight increase in the number of articles that presented some form of statistical technique was noticed for Brazilian journals (from 61.0 to 66.7%), whereas for international journals, a significant increase was observed (65.8 to 92.6%). In addition, a decrease in the number of statistical errors was verified. The most commonly used statistical tests as well as the most frequent errors found in dental journals were assessed. Hopefully, this investigation will encourage dental educators to better plan the teaching of biostatistics, and to improve the statistical quality of submitted manuscripts.

Parametric mean survival time analysis in gastric cancer patients

BACKGROUND

The mean survival time (MS) has acquired increasing importance as an outcome indicator for patient care and technology assessment. The authors use lifelong follow-up data from gastric cancer patients to study whether the MS is predictable from 5-year follow-up information based on 2 parametric models.

METHODS

The authors used 3597 gastric cancer patients operated on between 1950 and 1969 to create 50 groups. For each group, the disease-related survival curve (DRSC) was estimated from the 5-year follow-up data using the Boag model. The MS for the group was then estimated by combining the DRSC with the survival curve for the age and sex-matched contemporaries (control group) based on the competing risk model. Alternatively, it was estimated by using the DRSC and the MS for the control group (the survival limit model). These predicted MS values were compared with the full follow-up observations.

RESULTS

Although individual prediction errors varied depending on the group size (63 to 3597 patients) and the length of MS (0.3 to 20.2 years), the mean prediction errors were reasonably small; the survival limit model overestimated MS by 4.7% (95% confidence interval [CI], 1.6 to 7.8) and the competing risk model by 3.2% (95% CI, 0.1 to 6.5).

CONCLUSIONS

MS for gastric cancer patients is parametrically predictable from 5-year follow-up data. This analysis should be applicable to other diseases showing log-normal failure time distributions.

Enabling meta-analysis in systematic reviews on carpal tunnel syndrome

Possible solutions to the problems of clinical heterogeneity of outcome measures and inadequate reporting of results for randomized controlled trials (RCTs) on carpal tunnel syndrome (CTS) are presented. Meta-analysis was impeded by these problems in 2 systematic reviews concerning conservative and surgical treatment options for CTS. A solution to the problem of inadequate data presentation is to add explicit information on minimal requirements with regard to data presentation to guidelines for the reporting of studies. To resolve the problem of clinical heterogeneity of the outcomes there should be consensus on the (validated) outcomes that should be used in RCTs. For CTS there is little evidence available on the reliability, validity, and responsiveness to change of the commonly used outcomes in RCTs. Resolving both problems will increase the comparability of RCTs, enabling the calculation of a pooled estimate of effect in a meta-analysis.

Systematic reviews: gatekeepers of nursing knowledge

The past few decades have seen a considerable increase in the number of available health care products and interventions. This growth has been matched by a similar expansion in the health care literature. As a result of these factors, the demand for evidence to support practice is growing, but finding the best evidence is becoming increasingly difficult. In response, the use of systematic reviews is increasing and they are starting to replace the primary research as the basis for health care decisions. To date, these reviews have focused predominantly on effectiveness and so have been limited to randomized controlled trials. As a result of this, the interpretive, observational and descriptive research methods that are utilized by nursing have commonly been either excluded from the review or are classified as 'low level' evidence. To address this, nursing must participate in the development of systematic review methods that better answer the questions posed by the profession.

Randomised controlled trial of cardiotocography versus Doppler auscultation of fetal heart at admission in labour in low risk obstetric population

OBJECTIVE

To compare the effect of admission cardiotocography and Doppler auscultation of the fetal heart on neonatal outcome and levels of obstetric intervention in a low risk obstetric population.

DESIGN

Randomised controlled trial.

SETTING

Obstetric unit of teaching hospital

PARTICIPANTS

Pregnant women who had no obstetric complications that warranted continuous monitoring of fetal heart rate in labour.

INTERVENTION

Women were randomised to receive either cardiotocography or Doppler auscultation of the fetal heart when they were admitted in spontaneous uncomplicated labour.

MAIN OUTCOME MEASURES

The primary outcome measure was umbilical arterial metabolic acidosis. Secondary outcome measures included other measures of condition at birth and obstetric intervention.

RESULTS

There were no significant differences in the incidence of metabolic acidosis or any other measure of neonatal outcome among women who remained at low risk when they were admitted in labour. However, compared with women who received Doppler auscultation, women who had admission cardiotocography were significantly more likely to have continuous fetal heart rate monitoring in labour (odds ratio 1.49, 95% confidence interval 1.26 to 1.76), augmentation of labour (1.26, 1.02 to 1.56), epidural analgesia (1.33, 1.10 to 1.61), and operative delivery (1.36, 1.12 to 1.65).

CONCLUSIONS

Compared with Doppler auscultation of the fetal heart, admission cardiotocography does not benefit neonatal outcome in low risk women. Its use results in increased obstetric intervention, including operative delivery.

Systematic reviews of nursing research

As a result of the rapid advances in technology and the large volume of literature, healthcare decisions have become more complex. In response to this, increasing emphasis has been placed on basing these decisions on the best available research evidence. Systematic reviews are now accepted as the most reliable way by which this large volume of research evidence can be managed. These reviews follow the same principles expected of any research endeavour. This includes documentation of methods prior to commencement, comprehensive search to identify all studies on the topic, and the use of rigorous methods for the appraisal, collection and synthesis of data. On completion of the review, the methods used are reported to allow its validity to be evaluated by end users of the evidence.

Statistical reviewing policies of medical journals: caveat lector?

OBJECTIVE

To describe the current policies regarding statistical review of clinical research in biomedical journals.

DESIGN

Cross-sectional survey.

PARTICIPANTS

Editors of biomedical journals that publish original clinical research.

MEASUREMENTS

General policies on statistical review, types of persons used for statistical reviewing, compensation of statistical reviewers, percentage of articles subject to such review, percentage of time statistical review makes an important difference, journal circulation, and selectivity.

MAIN RESULTS

Of 171 journals, 114 (67%) responded to the survey. About one third of journals had policies that guaranteed statistical review for all accepted manuscripts. In approximately half of the journals, articles were sent for statistical review at the discretion of the editor. There was some evidence that statistical review policies differed between journals of different circulation size. In journals in the top quartile of circulation (> 25,000) the probability of definitely having a statistical review before an acceptance decision was 52%, but it was only 27% in journals in the lower three quartiles (p = .09). The probability of a statistical consultant on staff ranged from 31% in the bottom quarter, to 58% in the middle two, to 82% in the highest quarter (p < .001). Editors judged that statistical review resulted in an important change in a manuscript about half of the time.

CONCLUSIONS

Except in the largest circulation medical journals, the probability of formal methodologic review of original clinical research is fairly low. As readers and researchers depend on the journals to assess the validity of the statistical methods and logic used in published reports, this is potentially a serious problem. This situation may exist because the cost of such statistical review can be considerable, and because finding appropriate reviewers can be difficult. It may also exist partly because editors or publishers may not regard such review as important. The professions of medical publishing, statistics, epidemiology, and other quantitative disciplines should work together to address this problem.

Are community health interventions evaluated appropriately? A review of six journals

OBJECTIVES

To determine if Randomized Controlled Trial (RCT) methodology was used appropriately in community health, we: (1) determined the proportion of non-randomized studies that should have been RCTs, and (2) assessed the quality of the RCTs.

METHODS

The 1992 issues of six community health journals were manually searched. Intervention studies were analyzed. Studies that did not use randomization were analyzed for feasibility and practicality of RCT methods; RCTs were analyzed for quality using a checklist. RCTs were compared with community health RCTs from The New England Journal of Medicine. The proportion of studies meeting each criterion was determined.

RESULTS

Fourteen percent of 603 studies were interventions and 4% were RCTs. Of those not using randomization, 42% should have. Mean RCT scores were significantly lower for the community health journals than for The New England Journal of Medicine. Many criteria important to quality scored poorly.

CONCLUSIONS

RCTs are underused and lack methodologic rigor in community health. Conclusions regarding the effectiveness of interventions are therefore suspect.

Ten rules for reading clinical research reports

This was not a scientific assessment of the scientific quality of the papers published by The American Journal of Surgery. It was an informal audit of the adequacy of the data analysis in the clinical research reports appearing in the 1987-1988 issues. As one who has devoted more than three decades to helping a great variety of people make sense of scientific data, I found the overall quality of data analysis in these papers to be above average for the medical literature; and yet, I found many instances of errors so serious as to render invalid the conclusions of the authors. My 10 proposed rules for reading clinical research reports constitute only an interim solution to a very worrisome problem. The real solution must come from the producers of and the gatekeepers for the medical literature.

The organization of statistical activities at medical research institutions

In this paper, we discuss the organizational structure for the conduct of statistical activities at medical Colleges and Universities. Here, we have particularly focused on two significant statistical activities, i.e., statistical consultation and research. The consulting service for medical researchers consists of practical statistical analysis, instruction on computer manipulation and software, response to reviewer's comments and statistical design of prospective studies. From our various experiences, we describe the actual implementation of statistical consultations for medical researchers. It has played an important role in supporting medical research. In addition, we also outline some research, with respect to new ways of applying statistics in medical science. This paper concludes that it may be practical for the existing computer center or department of medical informatics, in charge of the computing service, to conduct statistical activities until formal organizations are established at the academic institution. To realize the conduct of statistical activities by Department of Medical Informatics, it needs a team of biostatisticians, data analysts and computer personnel.

Statistical tests in medical research

Various propositions have been made to improve the statistical quality of medical journals: using statistical referees, promoting better collaboration between statisticians and researchers, and teaching of basic statistics to clinicians. The most frequent errors found in medical articles are misinterpretation of p-values or non-significant results and confusion between statistical and clinical significance, inappropriate use of tests requiring precise assumptions, inappropriate or not controlled multiple testing and particularly testing of post hoc hypotheses, and overemphasis on p-values. Many errors arising from the misinterpretation of results of statistical hypothesis tests, the basic principles of this methodology are emphasized, and the usual fallacies found in the medical literature are reviewed.

Statistics in medical journals: developments in the 1980s

This paper reviews changes in the use of statistics in medical journals during the 1980s. Aspects considered are research design, statistical analysis, the presentation of results, medical journal policy (including statistical refereeing), and the misuse of statistics. Despite some notable successes, the misuse of statistics in medical papers remains common.

A new model for studying microcirculatory changes during dermal wound healing

Intact blood supply by microcirculation to a wounded site is an indispensable prerequisite for normal tissue regeneration. However, microvascular changes taking place in the healing process of skin wounds are not understood due to the fact that only few models allow chronic in vivo studies on skin microcirculation. Therefore, we have modified the hairless mouse ear model with the purpose of a quantitative in vivo study of microhemodynamic changes throughout the healing process. Following the creation of a standardized skin wound on the ear of the homozygous hairless mouse (hr/hr), microvessel diameters, red blood cell velocities, wet weight, and leucocyte content of the ear tissue were determined. Surface area of the wound was assessed until complete closure was achieved. By repeated measurements at identical sites over the entire healing period, a distinct pattern of microvascular changes could be observed: microvessel diameters increased to a maximum a few days after wound creation, whereas red blood cell velocities reached their highest values at a later point in time and were still elevated after complete reepithelization of the wounds. Edema and leucocyte content of the ear tissue was most prominent in the early healing phase and gradually decreased to normal values thereafter. These results demonstrate changes of the microvasculature of the hairless mouse ear to injury, which are in accordance to other more indirect studies on this topic. Therefore, we conclude that the model presented is suitable for prolonged quantitative analysis of microcirculation during normal wound healing and may be used to assess microvascular changes taking place during wound healing in pathologically altered tissue.

Statistical aspects of research methodology

Many reports in the medical literature are flawed through poor presentation of numerical results, or, more seriously, by the use of inadequate or inappropriate statistical methodology. This article expands on the more complex methodological issues. The main emphasis is on discussing key aspects of experimental design, and the idea of reporting results in terms of confidence intervals rather than P values.

Statistical guidelines for The British Journal of Surgery

A set of statistical guidelines is presented which should be followed by authors preparing manuscripts for submission to The British Journal of Surgery. Simple points relating to presentation are explained in full, and more complex issues are highlighted and discussed in an accompanying review article.

How we should approach the future

There is little doubt that there are still major problems with the statistical aspects of published medical research. This paper argues that it is not sufficient to teach statistics to medical undergraduates, when the bulk of their courses are being taught by the very same individuals who continue to undertake and publish substandard research. Instead, any undergraduate teaching needs to advance in parallel with courses given to the clinical staff within medical schools.

Ten rules for reading clinical research reports

This was not a scientific assessment of the scientific quality of the papers published by The American Journal of Surgery. It was an informal audit of the adequacy of the data analysis in the clinical research reports appearing in the 1987-1988 issues. As one who has devoted more than three decades to helping a great variety of people make sense of scientific data, I found the overall quality of data analysis in these papers to be above average for the medical literature; and yet, I found many instances of errors so serious as to render invalid the conclusions of the authors. My 10 proposed rules for reading clinical research reports constitute only an interim solution to a very worrisome problem. The real solution must come from the producers of and the gatekeepers for the medical literature.