The misuse of statistics: concepts, tools, and a research agenda

Advocating for Generalizability: Accepting Inherent Variability in Translation of Animal Research Outcomes

Advancing scientific discovery requires investigators to embrace research practices that increase transparency and disclosure about materials, methods, and outcomes. Several research advocacy and funding organizations have produced guidelines and recommended practices to enhance reproducibility through detailed and rigorous research approaches; however, confusion around vocabulary terms and a lack of adoption of suggested practices have stymied successful implementation. Although reproducibility of research findings cannot be guaranteed due to extensive inherent variables in attempts at experimental repetition, the scientific community can advocate for generalizability in the application of data outcomes to ensure a broad and effective impact on the comparison of animals to translation within human research. This report reviews suggestions, based upon work with National Institutes of Health advisory groups, for improving rigor and transparency in animal research through aspects of experimental design, statistical assessment, and reporting factors to advocate for generalizability in the application of comparative outcomes between animals and humans.

Disclosing and Managing Non-Financial Conflicts of Interest in Scientific Publications

In the last decade, there has been increased recognition of the importance of disclosing and managing non-financial conflicts of interests to safeguard the objectivity, integrity, and trustworthiness of scientific research. While funding agencies and academic institutions have had policies for addressing non-financial interests in grant peer review and research oversight since the 1990s, scientific journals have been only recently begun to develop such policies. An impediment to the formulation of effective journal policies is that non-financial interests can be difficult to recognize and define. Journals can overcome this problem by providing guidance concerning the types of non-financial interests that should be disclosed, including direct research interests, direct professional interests, expert testimony, involvement in litigation, holding a leadership position in a non-governmental organization, providing technical or scientific advice to a non-governmental organization, and personal or professional relationships. The guidance should apply to authors, editors, and reviewers.

Common Statistical Methods and Reporting of Results in Medical Research

Statistical analysis is critical in medical research. The objective of this article is to summarize the appropriate use and reporting of commonly used statistical methods in medical research, on the basis of existing statistical guidelines and the authors’ experience in reviewing manuscripts, to provide recommendations for statistical applications and reporting.

Perceptions of plagiarism among PhDs across the sciences, engineering, humanities, and arts: Results from a national survey in Brazil

Plagiarism allegations are not rare in the history of science, and credit for prior work was and continues to be a source of disputes, involving notions of priority of discovery and of plagiarism. However, consensus over what constitutes plagiarism among scientists from different fields cannot be taken for granted. We conducted a national survey exploring perceptions of plagiarism among PhD holders registered in the database of the Brazilian National Council for Scientific and Technological Development (CNPq). This survey was sent to 143,405 PhD holders across the fields, including biologists, physicists, mathematicians, and engineers as well as linguists, philosophers, and anthropologists, with a 20% response rate. The results suggest that core principles about plagiarism are shared among this multidisciplinary community, thus corroborating Robert K. Merton's observations that concerns over plagiarism and priority disputes are not field specific. This study offers insight into the way plagiarism is perceived in the research community and sheds light on the problem in the context of international collaborative research networks. The data focus on a particular research system in Latin America, but, given the cultural similarities that bind most Latin-American nations, these results may be relevant to other PhD populations in the region and should provide an opportunity for comparison with studies from other emerging, non-Anglophone regions.

The three ghosts of medical AI: Can the black-box present deliver?

Our title alludes to the three Christmas ghosts encountered by Ebenezer Scrooge in A Christmas Carol, who guide Ebenezer through the past, present, and future of Christmas holiday events. Similarly, our article takes readers through a journey of the past, present, and future of medical AI. In doing so, we focus on the crux of modern machine learning: the reliance on powerful but intrinsically opaque models. When applied to the healthcare domain, these models fail to meet the needs for transparency that their clinician and patient end-users require. We review the implications of this failure, and argue that opaque models (1) lack quality assurance, (2) fail to elicit trust, and (3) restrict physician-patient dialogue. We then discuss how upholding transparency in all aspects of model design and model validation can help ensure the reliability and success of medical AI.

Understanding the literature: Complexity of statistical methods used in high-impact cardiothoracic surgery research

OBJECTIVE

Biostatistics are frequently used in research published in the domain of cardiothoracic surgery. The objective of this study was to describe the scope of statistical techniques reported in the literature and to highlight implications for editorial review and critical appraisal.

METHODS

Original research articles published between January and April 2017 in the Journal of Thoracic and Cardiovascular Surgery, Annals of Thoracic Surgery, and the European Journal of Cardio-Thoracic Surgery were examined. For each article, the statistical method(s) reported were recorded and categorized by complexity.

RESULTS

We reviewed 293 articles that reported 1068 statistical methods. The mean number of different statistical methods reported per article was 3.6 ± 1.9, with variation by subspecialty and journal. The most common statistical methods were contingency tables (in 59% of articles), t tests (49%), and survival methods (49%). Only 4% of articles used descriptive statistics alone. An introductory level of statistical knowledge was deemed sufficient for understanding 16% of articles, whereas for the remainder a higher level of knowledge would be needed.

CONCLUSIONS

Contemporary cardiothoracic surgery research frequently requires the use of complex statistical methods. This was evident across articles for all cardiothoracic surgical subspecialties as reported in 3 high-impact journals. Routine review of manuscript submissions by biostatisticians is needed to ensure the appropriate use and reporting of advanced statistical methods in cardiothoracic surgery research.

Research Pearls: Checklists and Flowcharts to Improve Research Quality

To instill quality in published clinical research, reporting guidelines, consisting of checklists and flowcharts, were developed to protect against reporting poorly designed research, and researchers should be aware of the available instruments and their appropriate use. With the popularity of synthetic reviews, meta-analyses, and systematic reviews, there is a greater need to assess risk of bias and study quality. This review highlights the most frequently used guidelines and checklists, risk-of-bias scales, and quality rating scales that can assist researchers with improving their research and its eventual publication.

Statistical errors in articles published in radiology journals

PURPOSE

We aimed to evaluate articles in radiology journals indexed in the Science Citation Index or Science Citation Index Expanded in terms of statistical errors. By this means, we aim to contribute to the production of high quality scientific publications.

METHODS

In this study, a total of 157 articles published in 2016-2017 in 20 radiology journals were reviewed randomly. Selected articles were evaluated for statistical errors regarding P values and statistical tests, and for errors in terminology and other errors related to interpretation. In addition, in order to examine whether the error rates of the articles published in the radiology journals differed according to the impact factor, the statistical errors were compared according to the impact factors of the journals.

RESULTS

Of the 157 articles published in radiology journals, 10 had no statistical errors, while 147 had at least one statistical error. The most frequently encountered error was "errors in summarizing data" with a rate of 66%. This was followed by "incorrect representation of P values" with a rate of 42%. The least frequently encountered error was "statistical symbol errors" with a rate of 3%. There was no statistically significant difference according to impact factors.

CONCLUSION

In conclusion, radiology journals, as do journals in different fields, include articles containing statistical errors. Even when the quality of the journal increases, there is no difference in these statistical error rates. In order to prevent statistical errors in manuscripts, there are responsibilities for both the researchers who conduct scientific studies and the editors who publish these studies in their journals. Researchers should have a basic statistical knowledge, and the editor must submit all manuscripts for a statistical review.

Is it time to revise the definition of research misconduct?

U.S. federal policy defines research misconduct as fabrication of data, falsification of data, or plagiarism (FFP). In recent years, some have argued or suggested that the definition of research misconduct should also include sexual harassment, sabotage, deceptive use of statistics, and failure to disclose a significant conflict of interest (COI). While the arguments for revising the definition of misconduct used by federal agencies to include misbehaviors other than FFP are not convincing at this point in time, the arguments for revising definitions used by other organizations, such as professional societies, universities, or journals, may be. Since these other organizations play an important role in promoting integrity in science and deterring unethical behavior, they may consider adopting definitions of misconduct that extend beyond FFP. Debates about the definition of research misconduct are a normal and healthy part of broader discussions about integrity in science and how best to promote it. These debates should continue even if the federal definition of misconduct remains unchanged.

Basics, common errors and essentials of statistical tools and techniques in anesthesiology research

The statistical portion is a vital component of any research study. The research methodology and the application of statistical tools and techniques have evolved over the years and have significantly helped the research activities throughout the globe. The results and inferences are not accurately possible without proper validation with various statistical tools and tests. The evidencebased anesthesia research and practice has to incorporate statistical tools in the methodology right from the planning stage of the study itself. Though the medical fraternity is well acquainted with the significance of statistics in research, there is a lack of in-depth knowledge about the various statistical concepts and principles among majority of the researchers. The clinical impact and consequences can be serious as the incorrect analysis, conclusions, and false results may construct an artificial platform on which future research activities are replicated. The present tutorial is an attempt to make anesthesiologists aware of the various aspects of statistical methods used in evidence-based research and also to highlight the common areas where maximum number of statistical errors are committed so as to adopt better statistical practices.

The statistical reporting quality of articles published in 2010 in five dental journals

OBJECTIVES

Statistical methods play an important role in medical and dental research. In earlier studies it has been observed that current use of methods and reporting of statistics are responsible for some of the errors in the interpretation of results. The aim of this study was to investigate the quality of statistical reporting in dental research articles.

METHODS

A total of 200 articles published in 2010 were analysed covering five dental journals: Journal of Dental Research, Caries Research, Community Dentistry and Oral Epidemiology, Journal of Dentistry and Acta Odontologica Scandinavica. Each paper underwent careful scrutiny for the use of statistical methods and reporting. A paper with at least one poor reporting item has been classified as 'problems with reporting statistics' and a paper without any poor reporting item as 'acceptable'.

RESULTS

The investigation showed that 18 (9%) papers were acceptable and 182 (91%) papers contained at least one poor reporting item.

CONCLUSIONS

The proportion of at least one poor reporting item in this survey was high (91%). The authors of dental journals should be encouraged to improve the statistical section of their research articles and to present the results in such a way that it is in line with the policy and presentation of the leading dental journals.

Education in the responsible conduct of research in psychology: methods and scope

In this mixed-method study of education in the responsible conduct of research (RCR) in psychology, phase one survey respondents (n = 141) reported that faculty and students were familiar with RCR standards and procedures to educate them were believed to be adequate. However, educational methods varied widely. In phase two, seven survey respondents completed in-depth interviews assessing RCR training and education and research review procedures. Educational methods through which RCR content was presented included the following ones: traditional (lectures), technical (web-based), and experiential (internships), but RCR was often minimally considered in the formal curriculum. Our results suggest that psychology training programs might benefit from more formal consideration of RCR education and training in the curriculum.

Bias in research

By writing scientific articles we communicate science among colleagues and peers. By doing this, it is our responsibility to adhere to some basic principles like transparency and accuracy. Authors, journal editors and reviewers need to be concerned about the quality of the work submitted for publication and ensure that only studies which have been designed, conducted and reported in a transparent way, honestly and without any deviation from the truth get to be published. Any such trend or deviation from the truth in data collection, analysis, interpretation and publication is called bias. Bias in research can occur either intentionally or unintentionally. Bias causes false conclusions and is potentially misleading. Therefore, it is immoral and unethical to conduct biased research. Every scientist should thus be aware of all potential sources of bias and undertake all possible actions to reduce or minimize the deviation from the truth. This article describes some basic issues related to bias in research.

Evidence-based practice in nephrology: critical appraisal of nephrology clinical research: were the correct statistical tests used?

Nephrologists rely on valid clinical studies to inform their health care decisions. Knowledge of simple statistical principles equips the prudent nephrologist with the skills that allow him or her to critically evaluate clinical studies and to determine the validity of the results. Important in this process is knowing when certain statistical tests are used appropriately and if their application in interpreting research data will most likely lead to the most robust or valid conclusions. The research team bears the responsibility for determining the statistical analysis during the design phase of the study and subsequently for carrying out the appropriate analysis. This will ensure that bias is minimized and "valid" results are reported. We have summarized the important caveats and components in correctly choosing a statistical test with a series of tables. With this format, we wish to provide a tool for the nephrologist/researcher that he or she can use when required to decide if an appropriate statistical analysis plan was implemented for any particular study. We have included in these tables the types of statistical tests that might be used best for analysis of different types of comparisons on small and on larger patient samples.

Usefulness of statistics for establishing evidence-based reproductive medicine

During the last decade, evidence-based medicine has been described as a paradigm shift in clinical practice, and as "the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients". Appropriate statistical methods for analyzing data are critical for the correct interpretation of the results in proof of the evidence. However, in the medical literature, these statistical methods are often incorrectly interpreted or misinterpreted, leading to serious methodological errors and misinterpretations. This review highlights several important aspects related to the design and statistical analysis for evidence-based reproductive medicine. First, we clarify the distinction between ratios, proportions, and rates, and then provide a definition of pregnancy rate. Second, we focus on a special type of bias called 'confounding bias', which occurs when a factor is associated with both the exposure and the disease but is not part of the causal pathway. Finally, we present concerns regarding misuse of statistical software or application of inappropriate statistical methods, especially in medical research.

A case for a duty to feed the hungry: GM plants and the third world

This article is concerned with a discussion of the plausibility of the claim that GM technology has the potential to provide the hungry with sufficient food for subsistence. Following a brief outline of the potential applications of GM in this context, a history of the green revolution and its impact will be discussed in relation to the current developing world agriculture situation. Following a contemporary analysis of malnutrition, the claim that GM technology has the potential to provide the hungry with sufficient nourishment will be discussed within the domain of moral philosophy to determine whether there exists a moral obligation to pursue this end if and only if the technology proves to be relatively safe and effective. By using Peter Singer's duty of moral rescue, I argue that we have a moral duty to assist the third world through the distribution of such GM plants. I conclude the paper by demonstrating that my argument can be supported by applying a version of the Precautionary Principle on the grounds that doing nothing might be worse for the current situation.

Ophthalmology and vision science research

As part of an ongoing series, this fourth article is the second of 2 related articles that deal with the basics of producing a high-quality manuscript. Although ophthalmology and vision science are the principal focus of this series, the general concepts essential to producing a quality manuscript are applicable to diverse fields of research. This article provides guidelines on how to present research findings in a structured form using the Introduction, Methods, Results, and Discussion (IMRaD) style. The components to be included, and equally important excluded, from each section are elucidated in detail. Commonly omitted or inaccurately or incompletely presented components of the manuscript, such as statistical analysis and references, are highlighted. The necessity to carefully refine manuscripts before submission is discussed. This article concludes by illustrating methods of appropriately dealing with referees' comments and resubmitting manuscripts. Although researchers may have to deal with rejection as part of the process, we hope that this series provides an easy-to-follow structure to maximize manuscript quality and improve the prospect of scientific publication.