Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians

Perceptions on the role of research integrity officers in French medical schools

Researchers, organizations, and governments are trying to foster research integrity. In France, the law recently permitted the appointment of research integrity officers (RIOs) in each university, to promote research integrity and handle misconducts. Since we assumed that having adequate bodies to deal with research integrity could foster research integrity, we wanted to understand how this might work more concretely. We interviewed 11 newly appointed RIOs in medical schools about how they perceive their role and cope with their responsibilities. We analyzed data following the Paillé and Muchielli's thematic analysis approach. The RIOs report a strong and interesting appropriation of concepts of research integrity, which allows them to warrant their role. Although they report that they did not seek their appointment, they show a real desire to cope with their responsibilities. They are willing to build a role which is currently poorly defined. They assert their legitimacy through their position and experience. They identify themselves with a preventive and corrective role, in an altruistic way. They emphasize their independent and collective role, congruent with other actors. The RIOs intend to be enablers of a responsible conduct of research. These results are encouraging about the potential impact of RIOs to foster research integrity.

Spatial patterns and associated factors of HIV testing and counselling (HTC) as a component of antenatal care services in Ethiopia

BACKGROUND

While HIV testing and counselling play a crucial role in preventing mother-to-child transmission, numerous pregnant women did not receive these services. Understanding the spatial variation of HIV testing and counselling and its associated factors during antenatal care in Ethiopia remains limited. Thus, this study was aimed at assessing the spatial patterns and factors associated with HIV testing and counselling during antenatal care visits in Ethiopia.

METHODS

A cross-sectional study design was employed with a two-stage stratified cluster sampling technique. A total of 2,789 women who gave birth in the two years prior to the survey and had at least one antenatal care visit were included in the study. Stata version 16 and ArcGIS version 10.8 software were used for analysis. A multilevel robust Poisson regression model was fitted to identify significantly associated factors since the prevalence of HIV testing and counselling was higher than 10%. A statistically significant association was declared based on multivariable multilevel robust Poisson regression analysis using an adjusted prevalence ratio with its 95% confidence interval at a p-value < 0.05. Spatial regression analysis was conducted, and the local coefficients of statistically significant spatial covariates were visualised.

RESULTS

In Ethiopia, the overall prevalence of HIV testing and counselling during antenatal care visits was 29.5% (95% CI: 27.8%, 31.2%). Significant spatial clustering was observed (Global Moran's I = 0.138, p-value <0.001). In the spatial regression analysis, high and comprehensive knowledge related to HIV, and comprehensive knowledge on the prevention of mother-to-child transmission were significant explanatory variables for the spatial variation of HIV testing and counselling. In the multivariable multilevel robust Poisson regression analysis, education, household wealth, media exposure, number of antenatal care visits, comprehensive knowledge on mother-to-child transmission, comprehensive knowledge on prevention of mother-to-child transmission, and region were significantly associated factors.

CONCLUSION

The prevalence of HIV testing and counselling during antenatal care visits was low. Empowering women through education, promoting mass media exposure, increasing numbers of antenatal care visits, and enhancing women's knowledge related to HIV and mother-to-child transmission by targeting cold spot areas could improve HIV testing and counselling service uptake among pregnant women in Ethiopia.

Evidence of questionable research practices in clinical prediction models

BACKGROUND

Clinical prediction models are widely used in health and medical research. The area under the receiver operating characteristic curve (AUC) is a frequently used estimate to describe the discriminatory ability of a clinical prediction model. The AUC is often interpreted relative to thresholds, with "good" or "excellent" models defined at 0.7, 0.8 or 0.9. These thresholds may create targets that result in "hacking", where researchers are motivated to re-analyse their data until they achieve a "good" result.

METHODS

We extracted AUC values from PubMed abstracts to look for evidence of hacking. We used histograms of the AUC values in bins of size 0.01 and compared the observed distribution to a smooth distribution from a spline.

RESULTS

The distribution of 306,888 AUC values showed clear excesses above the thresholds of 0.7, 0.8 and 0.9 and shortfalls below the thresholds.

CONCLUSIONS

The AUCs for some models are over-inflated, which risks exposing patients to sub-optimal clinical decision-making. Greater modelling transparency is needed, including published protocols, and data and code sharing.

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. Scientists are granted considerable latitude to selectively use and report the hypotheses, variables and analyses that create the most positive, coherent and attractive story while suppressing those that are negative or inconvenient. This creates a risk of bias that can lead to scientists fooling themselves and fooling others. Preregistration involves declaring a research plan (for example, hypotheses, design and statistical analyses) in a public registry before the research outcomes are known. Preregistration (1) reduces the risk of bias by encouraging outcome-independent decision-making and (2) increases transparency, enabling others to assess the risk of bias and calibrate their confidence in research outcomes. In this Perspective, we briefly review the historical evolution of preregistration in medicine, psychology and other domains, clarify its pragmatic functions, discuss relevant meta-research, and provide recommendations for scientists and journal editors.

Keys for successful publication in Eur Ann Otorhinolaryngol Head Neck Dis: A STROBE analysis of peer reviews of articles submitted in 2020-2021

OBJECTIVE

To evaluate reviewing and editorial decision for articles submitted to the European Annals of Otorhinolaryngology Head & Neck Diseases.

MATERIALS AND METHODS

A retrospective analysis was made of reviewers' comments on 1,133 scientific articles (700 original articles, 96 literature reviews, and 337 case reports), originating from 69 countries, consecutively submitted on-line between January 1st, 2020 and December 31st, 2021. The main objective was to document the acceptance rate and decision time. Accessory objectives were to synthesize the main comments and to screen for correlations between acceptance and the main characteristics of first authors, articles and reviewers' comments.

RESULTS

In total, 4.1% of submitted articles were accepted. Median decision time differed significantly (P<0.0001), at 1 month in case of refusal and 4 months in case of acceptance. Reviewers mentioned failure to adhere to the journal's authors' guide, to use the appropriate EQUATOR guidelines and to adopt the recommended P<0.005 significance threshold in 94.8%, 54.2%, and 39.9% of cases, respectively. On multivariate analysis, 3 variables significantly impacted acceptance, which increased from 1.3% to 44.6% (P<0.0001) when an appropriate EQUATOR guideline was used and from 0.3% to 57.4% (P<0.0001) when the significance threshold was set at P<0.005, and decreased from 10.5% to 1.1% (P=0.0001) when the article did not originate from a French-speaking country (member of the Francophonie organization).

CONCLUSION

Adhesion to modern scientific medical writing rules increased acceptance rates for articles in the European Annals of Otorhinolaryngology Head & Neck Diseases. Teaching modern scientific medical writing needs to be enhanced in otorhinolaryngology.

Collaborative biostatistics and epidemiology in academic medical centres: A survey to assess relationships with health researchers and ethical implications

The role of collaborative biostatisticians and epidemiologists in academic medical centers and how their degree type, supervisor type, and sex influences recognition and feelings of respect is poorly understood. We conducted a cross-sectional survey of self-identified biostatisticians and epidemiologists working in academic medical centers in the US or Canada. The survey was sent to 341 contacts at 125 institutions who were asked to forward the survey invitation to faculty and staff at their institution and posted on Community sections of the American Statistical Association website. Participants were asked a variety of questions including if they felt pressured to produce specific results, whether they had intellectual and ethical freedom to pursue appropriate use of statistical methods in collaborative research, and if they felt their contributions were appropriately recognized by collaborators. We received responses from 314 biostatisticians or related methodologists. A majority were female (59%), had a doctorate degree (52%), and reported a statistician or biostatistician supervisor (69%). Overall, most participants felt valued by their collaborators, but that they did not have sufficient calendar time to meet deadlines. Doctoral-level participants reported more autonomy in their collaborations than master's level participants. Females were less likely to feel recognized and respected compared to males. The survey results suggest that while most respondents felt valued by their collaborators, they have too many projects and need more time to critically review research. Further research is needed to understand why response differs by sex and how these responses fluctuate over time.

How to write statistical analysis section in medical research

Reporting of statistical analysis is essential in any clinical and translational research study. However, medical research studies sometimes report statistical analysis that is either inappropriate or insufficient to attest to the accuracy and validity of findings and conclusions. Published works involving inaccurate statistical analyses and insufficient reporting influence the conduct of future scientific studies, including meta-analyses and medical decisions. Although the biostatistical practice has been improved over the years due to the involvement of statistical reviewers and collaborators in research studies, there remain areas of improvement for transparent reporting of the statistical analysis section in a study. Evidence-based biostatistics practice throughout the research is useful for generating reliable data and translating meaningful data to meaningful interpretation and decisions in medical research. Most existing research reporting guidelines do not provide guidance for reporting methods in the statistical analysis section that helps in evaluating the quality of findings and data interpretation. In this report, we highlight the global and critical steps to be reported in the statistical analysis of grants and research articles. We provide clarity and the importance of understanding study objective types, data generation process, effect size use, evidence-based biostatistical methods use, and development of statistical models through several thematic frameworks. We also provide published examples of adherence or non-adherence to methodological standards related to each step in the statistical analysis and their implications. We believe the suggestions provided in this report can have far-reaching implications for education and strengthening the quality of statistical reporting and biostatistical practice in medical research.

ALL-IN meta-analysis: breathing life into living systematic reviews

Science is justly admired as a cumulative process ("standing on the shoulders of giants"), yet scientific knowledge is typically built on a patchwork of research contributions without much coordination. This lack of efficiency has specifically been addressed in clinical research by recommendations for living systematic reviews and against research waste. We propose to further those recommendations with ALL-IN meta-analysis: Anytime Live and Leading INterim meta-analysis. ALL-IN provides statistical methodology for a meta-analysis that can be updated at any time-reanalyzing after each new observation while retaining type-I error guarantees, live-no need to prespecify the looks, and leading-in the decisions on whether individual studies should be initiated, stopped or expanded, the meta-analysis can be the leading source of information. We illustrate the method for time-to-event data, showing how synthesizing data at interim stages of studies can increase efficiency when studies are slow in themselves to provide the necessary number of events for completion. The meta-analysis can be performed on interim data, but does not have to. The analysis design requires no information about the number of patients in trials or the number of trials eventually included. So it can breathe life into living systematic reviews, through better and simpler statistics, efficiency, collaboration and communication.

Misinformation: an empirical study with scientists and communicators during the COVID-19 pandemic

OBJECTIVES

To study the experiences and views within the health science community regarding the spread and prevention of science misinformation within and beyond the setting of the COVID-19 pandemic.

METHODS

An exploratory study with an empirical ethics approach using qualitative interviews with Australians who produce, communicate and study health science research.

RESULTS

Key elements that participants considered might facilitate misinformation included: the production of low-quality, fraudulent or biased science research; inadequate public access to high-quality research; insufficient public reading of high-quality research. Strategies to reduce or prevent misinformation could come from within the academic community, academic and lay media publishing systems, government funders and educators of the general public. Recommended solutions from within the scientific community included: rewarding research translation, encouraging standardised study design, increasing use of automated quality assessment tools, mandating study protocol registration, transparent peer review, facilitating wider use of open access and use of newer technologies to target public audiences. There was disagreement over whether preprints were part of the problem or part of the solution.

CONCLUSIONS

There is concern from within the health science community about systemic failings that might facilitate the production and spread of false or misleading science information. We advocate for further research into ways to minimise the production and spread of misinformation about COVID-19 and other science crises in the future.

Toward more rigorous and informative nutritional epidemiology: The rational space between dismissal and defense of the status quo

To date, nutritional epidemiology has relied heavily on relatively weak methods including simple observational designs and substandard measurements. Despite low internal validity and other sources of bias, claims of causality are made commonly in this literature. Nutritional epidemiology investigations can be improved through greater scientific rigor and adherence to scientific reporting commensurate with research methods used. Some commentators advocate jettisoning nutritional epidemiology entirely, perhaps believing improvements are impossible. Still others support only normative refinements. But neither abolition nor minor tweaks are appropriate. Nutritional epidemiology, in its present state, offers utility, yet also needs marked, reformational renovation. Changing the status quo will require ongoing, unflinching scrutiny of research questions, practices, and reporting-and a willingness to admit that "good enough" is no longer good enough. As such, a workshop entitled "Toward more rigorous and informative nutritional epidemiology: the rational space between dismissal and defense of the status quo" was held from July 15 to August 14, 2020. This virtual symposium focused on: (1) Stronger Designs, (2) Stronger Measurement, (3) Stronger Analyses, and (4) Stronger Execution and Reporting. Participants from several leading academic institutions explored existing, evolving, and new better practices, tools, and techniques to collaboratively advance specific recommendations for strengthening nutritional epidemiology.

Methodology over metrics: current scientific standards are a disservice to patients and society

Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise

BACKGROUND

Researchers often misinterpret and misrepresent statistical outputs. This abuse has led to a large literature on modification or replacement of testing thresholds and P-values with confidence intervals, Bayes factors, and other devices. Because the core problems appear cognitive rather than statistical, we review some simple methods to aid researchers in interpreting statistical outputs. These methods emphasize logical and information concepts over probability, and thus may be more robust to common misinterpretations than are traditional descriptions.

METHODS

We use the Shannon transform of the P-value p, also known as the binary surprisal or S-value s = -log2(p), to provide a measure of the information supplied by the testing procedure, and to help calibrate intuitions against simple physical experiments like coin tossing. We also use tables or graphs of test statistics for alternative hypotheses, and interval estimates for different percentile levels, to thwart fallacies arising from arbitrary dichotomies. Finally, we reinterpret P-values and interval estimates in unconditional terms, which describe compatibility of data with the entire set of analysis assumptions. We illustrate these methods with a reanalysis of data from an existing record-based cohort study.

CONCLUSIONS

In line with other recent recommendations, we advise that teaching materials and research reports discuss P-values as measures of compatibility rather than significance, compute P-values for alternative hypotheses whenever they are computed for null hypotheses, and interpret interval estimates as showing values of high compatibility with data, rather than regions of confidence. Our recommendations emphasize cognitive devices for displaying the compatibility of the observed data with various hypotheses of interest, rather than focusing on single hypothesis tests or interval estimates. We believe these simple reforms are well worth the minor effort they require.

A review of the literature on ethical issues related to scientific authorship

The article at hand presents the results of a literature review on the ethical issues related to scientific authorship. These issues are understood as questions and/or concerns about obligations, values or virtues in relation to reporting, authorship and publication of research results. For this purpose, the Web of Science core collection was searched for English resources published between 1945 and 2018, and a total of 324 items were analyzed. Based on the review of the documents, ten ethical themes have been identified, some of which entail several ethical issues. Ranked on the basis of their frequency of occurrence these themes are: 1) attribution, 2) violations of the norms of authorship, 3) bias, 4) responsibility and accountability, 5) authorship order, 6) citations and referencing, 7) definition of authorship, 8) publication strategy, 9) originality, and 10) sanctions. In mapping these themes, the current article explores major ethical issue and provides a critical discussion about the application of codes of conduct, various understandings of culture, and contributing factors to unethical behavior.

Combined treatments with metformin and phosphodiesterase inhibitors alleviate nonalcoholic fatty liver disease in high-fat diet fed rats: a comparative study

Nonalcoholic fatty liver disease (NAFLD) is an excessive accumulation of fats in the liver resulting in hepatic inflammation and fibrous tissue formation along with insulin resistance. This study was designed to investigate the possible protective effects of metformin alone and in combination with different phosphodiesterase inhibitors (PDEIs). Rats were fed a high-fat diet (HFD) for 16 weeks to induce NAFLD. Starting from week 12, rats received metformin alone or in combination with pentoxifylline, cilostazol, or sildenafil. HFD administration resulted in hepatic steatosis and inflammation in rats. In addition, liver index, body composition index, activities of liver enzymes, and serum lipids deviated from normal. Further, significant elevations were recorded compared to control in terms of serum glucose, insulin, and HOMA-IR (homeostasis model assessment index for insulin resistance), oxidative stress parameters, hepatic TNF-α and NF-κB gene expression, and iNOS protein expression. Rats treated with metformin showed a significant improvement in the aforementioned parameters. However, the addition of pentoxifylline to metformin treatment synergized its action and produced a fortified effect against HFD-induced NAFLD better than other PDEIs. Data from this study indicated that combined treatment of metformin and pentoxifylline had the most remarkable ameliorated effects against HFD-induced NAFLD; further clinical investigations are needed to approve PDEIs for NAFLD treatment.

Comprehensive survey among statistical members of medical ethics committees in Germany on their personal impression of completeness and correctness of biostatistical aspects of submitted study protocols

OBJECTIVES

To assess biostatistical quality of study protocols submitted to German medical ethics committees according to personal appraisal of their statistical members.

DESIGN

We conducted a web-based survey among biostatisticians who have been active as members in German medical ethics committees during the past 3 years.

SETTING

The study population was identified by a comprehensive web search on websites of German medical ethics committees.

PARTICIPANTS

The final list comprised 86 eligible persons. In total, 57 (66%) completed the survey.

QUESTIONNAIRE

The first item checked whether the inclusion criterion was met. The last item assessed satisfaction with the survey. Four items aimed to characterise the medical ethics committee in terms of type and location, one item asked for the urgency of biostatistical training addressed to the medical investigators. The main 2×12 items reported an individual assessment of the quality of biostatistical aspects in the submitted study protocols, while distinguishing studies according to the German Medicines Act (AMG)/German Act on Medical Devices (MPG) and studies non-regulated by these laws.

PRIMARY AND SECONDARY OUTCOME MEASURES

The individual assessment of the quality of biostatistical aspects corresponds to the primary objective. Thus, participants were asked to complete the sentence 'In x% of the submitted study protocols, the following problem occurs', where 12 different statistical problems were formulated. All other items assess secondary endpoints.

RESULTS

For all biostatistical aspects, 45 of 49 (91.8%) participants judged the quality of AMG/MPG study protocols much better than that of 'non-regulated' studies. The latter are in median affected 20%-60% more often by statistical problems. The highest need for training was reported for sample size calculation, missing values and multiple comparison procedures.

CONCLUSIONS

Biostatisticians being active in German medical ethics committees classify the biostatistical quality of study protocols as low for 'non-regulated' studies, whereas quality is much better for AMG/MPG studies.

Should research misconduct be criminalized?

For more than 25 years, research misconduct (research fraud) is defined as fabrication, falsification, or plagiarism (FFP)—although other research misbehaviors have been also added in codes of conduct and legislations. A critical issue in deciding whether research misconduct should be subject to criminal law is its definition, because not all behaviors labeled as research misconduct qualifies as serious crime. But assuming that all FFP is fraud and all non-FFP not is far from obvious. In addition, new research misbehaviors have recently been described, such as prolific authorship, and fake peer review, or boosted such as duplication of images. The scientific community has been largely successful in keeping criminal law away from the cases of research misconduct. Alleged cases of research misconduct are usually looked into by committees of scientists usually from the same institution or university of the suspected offender in a process that often lacks transparency. Few countries have or plan to introduce independent bodies to address research misconduct; so for the coming years, most universities and research institutions will continue handling alleged research misconduct cases with their own procedures. A global operationalization of research misconduct with clear boundaries and clear criteria would be helpful. There is room for improvement in reaching global clarity on what research misconduct is, how allegations should be handled, and which sanctions are appropriate.

Messaging in Biological Psychiatry: Misrepresentations, Their Causes, and Potential Consequences

Most experts in the field of psychiatry recognize that neuroscience advances have yet to be translated into clinical practice. The main message delivered to laypeople, however, is that mental disorders are brain diseases cured by scientifically designed medications. Here we describe how this misleading message is generated. We summarize the academic studies describing how biomedical observations are often misrepresented in the scientific literature through various forms of data embellishment, publication biases favoring initial and positive studies, improper interpretations, and exaggerated conclusions. These misrepresentations also affect biological psychiatry and are spread through mass media documents. Exacerbated competition, hyperspecialization, and the need to obtain funding for research projects might drive scientists to misrepresent their findings. Moreover, journalists are unaware that initial studies, even when positive and promising, are inherently uncertain. Journalists preferentially cover them and almost never inform the public when those studies are disconfirmed by subsequent research. This explains why reductionist theories about mental health often persist in mass media even though the scientific claims that have been put forward to support them have long been contradicted. These misrepresentations affect the care of patients. Indeed, studies show that a neuro-essentialist conceptualization of mental disorders negatively affects several aspects of stigmatization, reduces the chances of patients' healing, and overshadows psychotherapeutic and social approaches that have been found effective in alleviating mental suffering. Public information about mental health should avoid these reporting biases and give equal consideration to the biological, psychological, and social aspects of mental health.

[The role of biostatistics in institutional review boards]

Research in humans is associated with risks. These risks are only justifiable if an independent institutional review board (IRB) has evaluated the planned research in terms of scientific integrity. Only scientifically sound research can be considered ethical. A biostatistician should be a member of the IRB to assure adequate evaluation of fundamental topics like design, sample size estimation, and statistical analysis of the study.This paper presents core biostatistical concepts following the current guidelines of the International Council of Harmonization (ICH E6 and ICH E9). We discuss important pitfalls based on examples from published clinical trials. Furthermore, we discuss new concepts like estimands and their relevance for biostatisticians working in IRBs. Finally, we discuss the role of biostatisticians in IRBs and present thoughts on the way they should be trained.

Bioethical Issues in Biostatistical Consulting Study: Additional Findings and Concerns

OBJECTIVES

To determine the frequency and severity of 17 requests for inappropriate analysis and reporting of data that biostatisticians received from researchers in statistical consultations, as indirectly experienced (i.e., either heard about or observed being asked of other biostatisticians).

METHODS

A randomly drawn sample of 522 members of the American Statistical Association who self-identified as consulting biostatisticians were selected to participate in an online survey with the Bioethical Issues in Biostatistical Consulting Questionnaire, specifically developed for this study.

RESULTS

Of the 522 consulting biostatisticians contacted, 390 (74.7%) completed the survey. The top 4 most frequently reported indirectly experienced inappropriate requests rated as "high severity" by at least 50% of the respondents were as follows: 1) proposing a study with a flawed design, including insufficient power; 2) setting aside values when the outcome turns on a few outliers; 3) reporting results of data analysis from only subsets of the data; and 4) overstating the statistical findings well beyond what the data support so that readers are misled.

CONCLUSIONS

This article is a follow-up report to our recently published article on 18 directly experienced inappropriate and/or unethical requests by biomedical researchers of their consulting biostatisticians. These additional survey findings from the Bioethical Issues in Biostatistical Consulting Questionnaire on 17 indirectly experienced inappropriate requests show the following: 1) the widespread nature of inappropriate, if not, frankly unethical requests made by biomedical researchers of their consulting biostatisticians and 2) the urgent need to develop and offer educational programs for biomedical researchers in training to correct these behaviors-be they intentional or accidental. This report speaks to the urgency for developing training programs for new and existing researchers to reduce the frequency of these inappropriate bioethical requests during biostatistical consultations.

KNOWLEDGE TRANSFER STATEMENT

The findings from this U.S. national survey of biostatisticians on inappropriate requests for data analyses by biomedical researchers strongly suggest a need for remedial actions, including 1) new educational modules for in-development and currently employed biomedical researchers, 2) improved institutional environments by research universities and companies regarding job and publication pressures, and 3) inclusion of a collaborating biostatistician as a working research team member from the early planning stages of all biomedical studies.