Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review

Self-reported checklists and quality scoring tools in radiomics: a meta-research

OBJECTIVE

To evaluate the use of reporting checklists and quality scoring tools for self-reporting purposes in radiomics literature.

METHODS

Literature search was conducted in PubMed (date, April 23, 2023). The radiomics literature was sampled at random after a sample size calculation with a priori power analysis. A systematic assessment for self-reporting, including the use of documentation such as completed checklists or quality scoring tools, was conducted in original research papers. These eligible papers underwent independent evaluation by a panel of nine readers, with three readers assigned to each paper. Automatic annotation was used to assist in this process. Then, a detailed item-by-item confirmation analysis was carried out on papers with checklist documentation, with independent evaluation of two readers.

RESULTS

The sample size calculation yielded 117 papers. Most of the included papers were retrospective (94%; 110/117), single-center (68%; 80/117), based on their private data (89%; 104/117), and lacked external validation (79%; 93/117). Only seven papers (6%) had at least one self-reported document (Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), or Checklist for Artificial Intelligence in Medical Imaging (CLAIM)), with a statistically significant binomial test (p < 0.001). Median rate of confirmed items for all three documents was 81% (interquartile range, 6). For quality scoring tools, documented scores were higher than suggested scores, with a mean difference of - 7.2 (standard deviation, 6.8).

CONCLUSION

Radiomic publications often lack self-reported checklists or quality scoring tools. Even when such documents are provided, it is essential to be cautious, as the accuracy of the reported items or scores may be questionable.

CLINICAL RELEVANCE STATEMENT

Current state of radiomic literature reveals a notable absence of self-reporting with documentation and inaccurate reporting practices. This critical observation may serve as a catalyst for motivating the radiomics community to adopt and utilize such tools appropriately, thereby fostering rigor, transparency, and reproducibility of their research, moving the field forward.

KEY POINTS

• In radiomics literature, there has been a notable absence of self-reporting with documentation. • Even if such documents are provided, it is critical to exercise caution because the accuracy of the reported items or scores may be questionable. • Radiomics community needs to be motivated to adopt and appropriately utilize the reporting checklists and quality scoring tools.

Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study

OBJECTIVES

The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting.

DESIGN

Methodological cross-sectional study.

METHODS

Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively.

RESULTS

We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037).

CONCLUSIONS

Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result.

STUDY REGISTRATION NUMBER

The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).

The Oral Health Statistics Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the Oral Health Statistics Guidelines. Drafts were circulated to the editors of 85 oral health journals and to task force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the task force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

The possible short-term of Nigella sativa - L in the management of adolescent polycystic ovarian syndrome: results of a randomized controlled trial

BACKGROUND

Polycystic ovary syndrome (PCOS) is the most common endocrine disorder in reproductive age and the most common cause of infertility due to anovulation. PCOS in adolescents is concerning. Nigella sativa is effective in improving gonadotropins and sex hormones. The current study was designed to investigate the effect of Nigella sativa supplementation on PCOS symptoms and their severity in adolescents.

METHODS

The current randomized clinical trial was conducted on 114 adolescents with PCOS who were referred to gynecologist offices and clinics in Gonabad, Iran from March 2022 to March 2023. Participants were randomly allocated to the intervention (Nigella sativa 1000 mg/day) and control (10 mg/day medroxyprogesterone from the 14th day of the cycle for 10 nights) groups. The study duration was 16 weeks. Ovarian volume (measured by ultrasound), anthropometric and blood pressure; serum testosterone, dehydroepiandrosterone (DHEA), dehydroepiandrosterone sulfate (DHEA-S), luteinizing hormone (LH), hirsutism severity (Ferriman-Gallwey score) levels were evaluated before and after the study.

RESULTS

Data from 103 participants (control group = 53, intervention group = 50) were analyzed. The mean age of participants was 17.0 (Interquartile range [IQR]:2.0). The mean difference in hirsutism score changes (p < 0.001), right (p = 0.002), and left (p = 0.010) ovarian volume, serum LH (p < 0.001) and testosterone (p = 0.001) were significantly higher in the intervention group compared to the control group. The frequency of oligomenorrhea, menometrorrhagia, and amenorrhea, were significantly reduced after the study in the intervention group compared to the control group (ps < 0.001).

CONCLUSIONS

Short-term Nigella sativa supplementation may be effective in reducing ovarian volume and improving hormonal balance, and menstrual irregularities in adolescents with PCOS. Further research and long-term studies are warranted to validate the potential therapeutic effects of Nigella sativa in adolescents with PCOS.

IRCT REGISTRATION NUMBER

IRCT20221017056209N1 Registration date: 2022-11-22.

The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: explanation and elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

Completeness of intervention reporting of nutrition-focused weight management interventions adjunct to metabolic and bariatric surgery: effect of the TIDieR checklist

Nutrition-focused interventions are essential to optimize the bariatric care process and improve health and weight outcomes over time. Clear and detailed reporting of these interventions in research reports is crucial for understanding and applying the findings effectively in clinical practice and research replication. Given the importance of reporting transparency in research, this study aimed to use the Template for Intervention Description and Replication (TIDieR) checklist to evaluate the completeness of intervention reporting in nutritional weight management interventions adjunct to metabolic and bariatric surgery (MBS). The secondary aim was to examine the factors associated with better reporting. A literature search in PubMed, PsychINFO, EMBASE, Scopus, and the Cochrane Controlled Register of Trials was conducted to include randomized controlled trials (RCT), quasi-RCTs and parallel group trials. A total of 22 trials were included in the final analysis. Among the TIDieR 12 items, 6.6 ± 1.9 items were fully reported by all studies. None of the studies completely reported all intervention descriptors. The main areas where reporting required improvement were providing adequate details of the materials and procedures of the interventions, intervention personalization, and intervention modifications during the study. The quality of intervention reporting remained the same after vs. before the release of the TIDieR guidelines. Receiving funds from industrial organizations (p = 0.02) and having the study recorded within a registry platform (p = 0.08) were associated with better intervention reporting. Nutritional weight management interventions in MBS care are still below the desirable standards for reporting. The present study highlights the need to improve adequate reporting of such interventions, which would allow for greater replicability, evaluation through evidence synthesis studies, and transferability into clinical practice.

The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement

BACKGROUND AND AIM

Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts.

METHODS

We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting.

RESULTS

Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts.

CONCLUSIONS

The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.

REFORMS: Consensus-based Recommendations for Machine-learning-based Science

Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and generalizability. These failures can hinder scientific progress, lead to false consensus around invalid claims, and undermine the credibility of ML-based science. ML methods are often applied and fail in similar ways across disciplines. Motivated by this observation, our goal is to provide clear recommendations for conducting and reporting ML-based science. Drawing from an extensive review of past literature, we present the REFORMS checklist (recommendations for machine-learning-based science). It consists of 32 questions and a paired set of guidelines. REFORMS was developed on the basis of a consensus of 19 researchers across computer science, data science, mathematics, social sciences, and biomedical sciences. REFORMS can serve as a resource for researchers when designing and implementing a study, for referees when reviewing papers, and for journals when enforcing standards for transparency and reproducibility.

Endorsement of reporting guidelines and clinical trial registration across Scopus-indexed rheumatology journals: a cross-sectional analysis

The purpose of this study was to investigate the instructions for authors of rheumatology journals and analyze their endorsement of reporting guidelines and clinical trial registration. Sixty rheumatology journals were selected by a research librarian and an investigator through the 2021 Scopus CiteScore tool. The instructions for authors' subsection of each journal was assessed to determine endorsement of study design-specific reporting guidelines or clinical trial registration. Descriptive statistics were calculated using R (version 4.2.1) and RStudio. Of the 58 journals analyzed, 34 (34/58; 59%) mentioned the EQUATOR Network: an online compendium of best practice reporting guidelines. The most commonly mentioned reporting guidelines were CONSORT with 44 journals (44/58; 75%), and PRISMA with 35 journals (35/58; 60%). The least mentioned guidelines were QUOROM with 56 journals not mentioning the guideline (56/58; 97%), and SRQR with 53 journals not mentioning the guideline (53/57, 93%). Clinical trial registration was required by 38 journals (38/58; 66%) and recommended by 8 journals (8/58; 14%). Our study found that endorsement of reporting guidelines and clinical trial registration within rheumatology journals was suboptimal with great room for improvement. Endorsement of reporting guidelines have shown to not only mitigate bias, but also improve research methodologies. Therefore, we recommend rheumatology journals broadly expand their endorsement of reporting guidelines and clinical trial registration to improve the quality of evidence they publish.

Research and mentorship in behavior analysis from a lens of cultural responsiveness and antiracism

Although scientific endeavors strive to be objective, they are the work of individuals whose unique perspectives and experiences influence their research and interpretations of the world and data. Much has been said and written lately about the need to embed cultural responsiveness in behavior analysis and the need to enhance diversity in the field. In fact, similar conversations are taking place in many areas of science. Despite the current buzz, many behavioral researchers may be left wondering what they can do or whether it is incumbent on them to act. For the field of behavior analysis to move toward adopting the values of diversity, equity, inclusion, and access, members of the scientific community must actively engage in behaviors that foster inclusive and safe learning environments for students, engage in collaborative work, and incorporate culturally responsive research and mentorship practices. This article will describe some current practices, showcase exemplars of culturally responsive research and mentorship, and provide resources for researchers and mentors.

Clinical performance of short fiber reinforced composite and glass hybrid on hypomineralized molars: A 36-month randomized split-mouth study

OBJECTIVES

This study aimed to assess the clinical performance outcome at 36 months of molars with molar incisor hypomineralization and carious lesions, treated with two different restorative approaches following selective caries removal.

METHODS

The children aged 6 to 12 years (18 female, 13 male) included in the study had at least two carious permanent first molars diagnosed with molar incisor hypomineralization. Sixty-two molars were restored in a split-mouth design. In all subjects, selective caries removal was performed so that caries was completely removed from the cavosurface walls and only soft dentin was left above the pulp chamber. Short fiber reinforced composite (SFRC; EverX Flow™) covered by micro-hybrid composite (G-Aenial® posterior composite) and Glass Hybrid (GH; Equia Forte® HT) were used as restorative materials. The restorations were evaluated according to modified United States Public Health Service (USPHS) criteria at baseline, 6, 12, 18, 24, and 36 month follow-ups.

RESULTS

During the 36-month follow-up, eight GH and four SFRC restorations failed. The clinical success of both restorations decreased statistically over time (p < 0.001 for both). When variables such as restoration type, sex, age, tooth type, and time were included in the model, the risk of failure of the restorations of the left lower first molar was statistically significantly higher than that of the left upper first molar (p < 0.002).

CONCLUSION

Direct composite restorations with SFRC and GH restorations perform similar clinical success with selective caries removal in the management of permanent molars affected by molar incisor hypomineralization.

CLINICAL SIGNIFICANCE

SFRC or GH restorations with similar clinical success might be preferred for the management of MIH-affected molars.

Sex reporting of cells used in cancer research: A systematic review

Sex and gender disparities in biomedical research have been emphasized to improve scientific knowledge applied for the health of both men and women. Despite sex differences in cancer incidence, prognosis, and responses to therapeutic agents, mechanistic explanations at molecular levels are far from enough. Recent studies suggested that cell sex is an important biological variable due to differences in sex chromosome gene expression and differences in events associated with developmental biology. The objective of this study was to analyze the reporting of sex of cells used in cancer research using articles published in Cancer Cell, Molecular Cancer, Journal of Hematology & Oncology, Journal for ImmunoTherapy of Cancer, and Cancer Research in 2020, and to examine whether there exists any sex bias. We found that the percentage of cells with sex notation in the article was 36.5%. Primary cells exhibited higher sex notation compared to cell lines. A higher percentage of female cells were used in cell cultures with sex notation. Also, sex-common cells omitted sex description more often compared to sex-specific cells. None of the cells isolated from embryo and esophagus reported the cell sex in the article. Our results indicate cell sex report in cancer research is limited to a small proportion of cells used in the study. These results call for acknowledging the sex of cells to increase the applicability of biomedical research discoveries.

Quality reporting of randomized controlled trials on SGLT2 inhibitors for heart failure: a comprehensive assessment

Randomised controlled trials (RCTs) provide clinicians with the best evidence of the effectiveness of an intervention, and complete and transparent trial reports help to critically assess and use trial results. The objective of our study was to assess the quality of reporting in RCTs of sodium-glucose co-transporter protein 2 (SGLT2) inhibitors for heart failure (HF) and identify factors associated with improved reporting quality. Two researchers conducted a comprehensive search in four databases (PubMed, Web of Science, EMBASE, and Cochrane). The quality of each report was assessed using a 25-point Overall Quality Score (OQS) based on the guidelines provided in the 2010 Consolidated Standards for Reporting of Trials (CONSORT) statement. We included a total of 58 relevant RCTs. The median OQS in the 2010 CONSORT statement was 15 (range 7.5-24). The missing items were primarily found in the 'Methods' and 'Results' sections of the 2010 CONSORT statement. Multivariate regression modeling revealed that a more recent publication year, high impact factor, and large sample size were significant predictors of OQS improvement. The findings suggest that the overall quality of reported RCTs of SGLT2 inhibitors in HF is unsatisfactory, which reduces their potential usefulness.

Risk of bias in exercise science: A systematic review of 340 studies

Risk of bias can contribute to irreproducible science and mislead decision making. Analyses of smaller subsections of the exercise science literature suggest many exercise science studies have unclear or high risk of bias. The current review (osf.io/jznv8) assesses whether this unclear or high risk of bias is more widespread in the exercise science literature and whether this bias has decreased since the publication of the 1996 Consolidated Standards of Reporting Trials (CONSORT) guidelines. We report significant reductions in selection, performance, detection, and reporting biases in 2020 compared with 1995 in the 340 of 5,451 studies assessed using the Cochrane Risk of Bias tool. Despite these improvements, most 2020 studies still had unclear or high risks of bias. These results underscore the need for methodological vigilance, adherence to reporting standards, and education on experimental bias. Factors contributing to these improvements, such advancements in education and journal requirements, remain uncertain.

Primary outcome reporting in clinical trials for older adults with depression

BACKGROUND

Findings from randomised controlled trials (RCTs) are synthesised through meta-analyses, which inform evidence-based decision-making. When key details regarding trial outcomes are not fully reported, knowledge synthesis and uptake of findings into clinical practice are impeded.

AIMS

Our study assessed reporting of primary outcomes in RCTs for older adults with major depressive disorder (MDD).

METHOD

Trials published between 2011 and 2021, which assessed any intervention for adults aged ≥65 years with a MDD diagnosis, and that specified a single primary outcome were considered for inclusion in our study. Outcome reporting assessment was conducted independently and in duplicate with a 58-item checklist, used in developing the CONSORT-Outcomes statement, and information in each RCT was scored as 'fully reported', 'partially reported' or 'not reported', as applicable.

RESULTS

Thirty-one of 49 RCTs reported one primary outcome and were included in our study. Most trials (71%) did not fully report over half of the 58 checklist items. Items pertaining to outcome analyses and interpretation were fully reported by 65% or more of trials. Items reported less frequently included: outcome measurement instrument properties (varied from 3 to 30%) and justification of the criteria used to define clinically meaningful change (23%).

CONCLUSIONS

There is variability in how geriatric depression RCTs report primary outcomes, with omission of details regarding measurement, selection, justification and definition of clinically meaningful change. Outcome reporting deficiencies may hinder replicability and synthesis efforts that inform clinical guidelines and decision-making. The CONSORT-Outcomes guideline should be used when reporting geriatric depression RCTs.

Health psychology and climate change: time to address humanity's most existential crisis

Climate change is an ongoing and escalating health emergency. It threatens the health and wellbeing of billions of people, through extreme weather events, displacement, food insecurity, pathogenic diseases, societal destabilisation, and armed conflict. Climate change dwarfs all other challenges studied by health psychologists. The greenhouse gas emissions driving climate change disproportionately originate from the actions of wealthy populations in the Global North and are tied to excessive energy use and overconsumption driven by the pursuit of economic growth. Addressing this crisis requires significant societal transformations and individual behaviour change. Most of these changes will benefit not only the stability of the climate but will yield significant public health co-benefits. Because of their unique expertise and skills, health psychologists are urgently needed in crafting climate change mitigation responses. We propose specific ways in which health psychologists at all career stages can contribute, within the spheres of research, teaching, and policy making, and within organisations and as private citizens. As health psychologists, we cannot sit back and leave climate change to climate scientists. Climate change is a health emergency that results from human behaviour; hence it is in our power and responsibility to address it.

The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review

BACKGROUND

Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine.

OBJECTIVE

We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies.

METHODS

We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility.

RESULTS

After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant.

CONCLUSIONS

The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

Quality of reporting of integrative Chinese and Western medicine intervention in randomized controlled trials of ulcerative colitis: a review

BACKGROUND

Integrative Chinese and Western medicine (ICWM) is commonly used for the treatment of ulcerative colitis (UC) in clinical practice. However, it is unclear whether the details of ICWM interventions, such as selection rationale, implementation design, and potential interactions, were adequately reported. Therefore, this study aimed to assess the quality of reporting in the ICWM interventional randomized controlled trials (RCTs) of UC and to identify the common problems if any.

METHODS

Through a search of 10 international electronic databases, we identified RCTs of UC with ICWM interventions published in English or Chinese from the inception date of each database up to 16 June 2023. Literature screening was strictly conducted based on the inclusion and exclusion criteria of the Population, Concept, and Context (PCC) framework. The general characteristics of the included studies were described. The quality of reporting was assessed according to three checklists, including the CONSORT (Consolidated Standards of Reporting Trials) with 36 items (except for one item 1b about abstract), the CONSORT for Abstracts (17 items), and a self-designed ICWM-related checklist (27 items covering design rationale, intervention details, outcome assessments, and analysis). The reporting scores of RCTs published before and after 2010 were compared.

RESULTS

A total of 1458 eligible RCTs were included. For the reporting compliance, the median score (interquartile ranges) of the CONSORT (72 score in total), the CONSORT for Abstract (34 score), and ICWM-related (54 score) items was 21 (18-25), 13 (12-15), and 18 (15-21), respectively. Although the time period comparisons showed that reporting quality of included publications improved significantly after the CONSORT 2010 issued (P < 0.01), more than 50% of items were evaluated as poor quality (reporting rate < 65%) among each checklist, especially in the CONSORT for Abstract and ICWM-specific items.

CONCLUSION

Although CONSORT appears to have enhanced the reporting of RCTs in UC, the quality of ICWM specifics is variable and in need of improvement. Reporting guidelines of the ICWM recommendations should be developed to improve their quality.

What is the quality of reporting in randomized controlled trials in spinal conditions

Purpose

Substandard quality across published randomized controlled trials (RCTs) is a major concern. Imperfect reporting has the potential to distort the evidence landscape and waste valuable health-care resources. In this study, we aim to assess the current quality of reporting in the field of spine using a modified version of the Consolidated Standards of Reporting Trials (CONSORT) checklist.

Materials and Methods

A list of published RCTs in the field of spine disease from January 1, 2013, to December 31, 2020, was built. Two reviewers scored the published RCTs against a modified CONSORT checklist. The mean adjusted CONSORT scores for each study, reporting category, and checklist item were calculated.

Results

The mean and median scores across all of the RCTs were 0.72 and 0.74 out of 1.00, respectively. The spectrum of scores was wide, ranging from 0.45 to 0.94. The reporting categories with the lowest score included randomization, blinding, and abstract. The items which were most under-reported included allocation sequence generation, type of randomization used, full trial protocol details, and abstract methodology. The inter-rater reliability between our reviewers was substantial (κ = 0.7, κ = 0.71).

Conclusion

Our findings correlate with only a moderate level of compliance to the CONSORT criteria on the quality of reporting for RCTs in spinal conditions. This is in line with previous reports on compliance, both within and outside the field of spinal conditions. Further continued and sustained efforts are still required to enhance the quality and consistency of RCT reporting, ultimately reducing health-care resource wastage and improving patient safety.

Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review

BACKGROUND

Guidelines, checklists, frameworks, and recommendations (GCFRs) related to preliminary studies serve as essential resources to assist behavioral intervention researchers in reporting findings from preliminary studies, but their impact on preliminary study reporting comprehensiveness is unknown. The purpose of this study was to conduct a scoping bibliometric review of recently published preliminary behavioral-focused intervention studies to (1) examine the prevalence of GCFR usage and (2) determine the associations between GCFR usage and reporting feasibility-related characteristics.

METHODS

A systematic search was conducted for preliminary studies of behavioral-focused interventions published between 2018 and 2020. Studies were limited to the top 25 journals publishing behavioral-focused interventions, text mined to identify usage of GCFRs, and categorized as either not citing GCFRs or citing ≥ 2 GCFRs (Citers). A random sample of non-Citers was text mined to identify studies which cited other preliminary studies that cited GCFRs (Indirect Citers) and those that did not (Never Citers). The presence/absence of feasibility-related characteristics was compared between Citers, Indirect Citers, and Never Citers via univariate logistic regression.

RESULTS

Studies (n = 4143) were identified, and 1316 were text mined to identify GCFR usage (n = 167 Citers). A random sample of 200 studies not citing a GCFR were selected and categorized into Indirect Citers (n = 71) and Never Citers (n = 129). Compared to Never Citers, Citers had higher odds of reporting retention, acceptability, adverse events, compliance, cost, data collection feasibility, and treatment fidelity (ORrange = 2.62-14.15, p < 0.005). Citers also had higher odds of mentioning feasibility in purpose statements, providing progression criteria, framing feasibility as the primary outcome, and mentioning feasibility in conclusions (ORrange = 6.31-17.04, p < 0.005) and lower odds of mentioning efficacy in purpose statements, testing for efficacy, mentioning efficacy in conclusions, and suggesting future testing (ORrange = 0.13-0.54, p < 0.05). Indirect Citers had higher odds of reporting acceptability and treatment fidelity (ORrange = 2.12-2.39, p < 0.05) but lower odds of testing for efficacy (OR = 0.36, p < 0.05) compared to Never Citers.

CONCLUSION

The citation of GCFRs is associated with greater reporting of feasibility-related characteristics in preliminary studies of behavioral-focused interventions. Researchers are encouraged to use and cite literature that provides guidance on design, implementation, analysis, and reporting to improve the comprehensiveness of reporting for preliminary studies.

Two decades of surgical randomized controlled trials: worldwide trends in volume and methodological quality

BACKGROUND

RCTs are essential in guiding clinical decision-making but are difficult to perform, especially in surgery. This review assessed the trend in volume and methodological quality of published surgical RCTs over two decades.

METHODS

PubMed was searched systematically for surgical RCTs published in 1999, 2009, and 2019. The primary outcomes were volume of trials and RCTs with a low risk of bias. Secondary outcomes were clinical, geographical, and funding characteristics.

RESULTS

Some 1188 surgical RCTs were identified, of which 300 were published in 1999, 450 in 2009, and 438 in 2019. The most common subspecialty in 2019 was gastrointestinal surgery (50.7 per cent). The volume of surgical RCTs increased mostly in Asia (61, 159, and 199 trials), especially in China (7, 40, and 81). In 2019, countries with the highest relative volume of published surgical RCTs were Finland and the Netherlands. Between 2009 and 2019, the proportion of RCTs with a low risk of bias increased from 14.7 to 22.1 per cent (P = 0.004). In 2019, the proportion of trials with a low risk of bias was highest in Europe (30.5 per cent), with the UK and the Netherlands as leaders in this respect.

CONCLUSION

The volume of published surgical RCTs worldwide remained stable in the past decade but their methodological quality improved. Considerable geographical shifts were observed, with Asia and especially China leading in terms of volume. Individual European countries are leading in their relative volume and methodological quality of surgical RCTs.

Methods for Heart Rate Variability Biofeedback (HRVB): A Systematic Review and Guidelines

Heart Rate Variability Biofeedback (HRVB) has been widely used to improve cardiovascular health and well-being. HRVB is based on breathing at an individual's resonance frequency, which stimulates respiratory sinus arrhythmia (RSA) and the baroreflex. There is, however, no methodological consensus on how to apply HRVB, while details about the protocol used are often not well reported. Thus, the objectives of this systematic review are to describe the different HRVB protocols and detect methodological concerns. PsycINFO, CINALH, Medline and Web of Science were searched between 2000 and April 2021. Data extraction and quality assessment were based on PRISMA guidelines. A total of 143 studies were finally included from any scientific field and any type of sample. Three protocols for HRVB were found: (i) "Optimal RF" (n = 37), each participant breathes at their previously detected RF; (ii) "Individual RF" (n = 48), each participant follows a biofeedback device that shows the optimal breathing rate based on cardiovascular data in real time, and (iii) "Preset-pace RF" (n = 51), all participants breathe at the same rate rate, usually 6 breaths/minute. In addition, we found several methodological differences for applying HRVB in terms of number of weeks, duration of breathing or combination of laboratory and home sessions. Remarkably, almost 2/3 of the studies did not report enough information to replicate the HRVB protocol in terms of breathing duration, inhalation/exhalation ratio, breathing control or body position. Methodological guidelines and a checklist are proposed to enhance the methodological quality of future HRVB studies and increase the information reported.

Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Modeling Studies: Development and Validation

BACKGROUND

The reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements.

OBJECTIVE

We aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies.

METHODS

We conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average κ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items.

RESULTS

In total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting.

CONCLUSIONS

Our findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies.

Evaluation of randomized controlled trials: a primer and tutorial for mental health researchers

BACKGROUND

Considered one of the highest levels of evidence, results of randomized controlled trials (RCTs) remain an essential building block in mental health research. They are frequently used to confirm that an intervention "works" and to guide treatment decisions. Given their importance in the field, it is concerning that the quality of many RCT evaluations in mental health research remains poor. Common errors range from inadequate missing data handling and inappropriate analyses (e.g., baseline randomization tests or analyses of within-group changes) to unduly interpretations of trial results and insufficient reporting. These deficiencies pose a threat to the robustness of mental health research and its impact on patient care. Many of these issues may be avoided in the future if mental health researchers are provided with a better understanding of what constitutes a high-quality RCT evaluation.

METHODS

In this primer article, we give an introduction to core concepts and caveats of clinical trial evaluations in mental health research. We also show how to implement current best practices using open-source statistical software.

RESULTS

Drawing on Rubin's potential outcome framework, we describe that RCTs put us in a privileged position to study causality by ensuring that the potential outcomes of the randomized groups become exchangeable. We discuss how missing data can threaten the validity of our results if dropouts systematically differ from non-dropouts, introduce trial estimands as a way to co-align analyses with the goals of the evaluation, and explain how to set up an appropriate analysis model to test the treatment effect at one or several assessment points. A novice-friendly tutorial is provided alongside this primer. It lays out concepts in greater detail and showcases how to implement techniques using the statistical software R, based on a real-world RCT dataset.

DISCUSSION

Many problems of RCTs already arise at the design stage, and we examine some avoidable and unavoidable "weak spots" of this design in mental health research. For instance, we discuss how lack of prospective registration can give way to issues like outcome switching and selective reporting, how allegiance biases can inflate effect estimates, review recommendations and challenges in blinding patients in mental health RCTs, and describe problems arising from underpowered trials. Lastly, we discuss why not all randomized trials necessarily have a limited external validity and examine how RCTs relate to ongoing efforts to personalize mental health care.

Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework

OBJECTIVE

To derive a comprehensive implementation framework for clinical AI models within hospitals informed by existing AI frameworks and integrated with reporting standards for clinical AI research.

MATERIALS AND METHODS

(1) Derive a provisional implementation framework based on the taxonomy of Stead et al and integrated with current reporting standards for AI research: TRIPOD, DECIDE-AI, CONSORT-AI. (2) Undertake a scoping review of published clinical AI implementation frameworks and identify key themes and stages. (3) Perform a gap analysis and refine the framework by incorporating missing items.

RESULTS

The provisional AI implementation framework, called SALIENT, was mapped to 5 stages common to both the taxonomy and the reporting standards. A scoping review retrieved 20 studies and 247 themes, stages, and subelements were identified. A gap analysis identified 5 new cross-stage themes and 16 new tasks. The final framework comprised 5 stages, 7 elements, and 4 components, including the AI system, data pipeline, human-computer interface, and clinical workflow.

DISCUSSION

This pragmatic framework resolves gaps in existing stage- and theme-based clinical AI implementation guidance by comprehensively addressing the what (components), when (stages), and how (tasks) of AI implementation, as well as the who (organization) and why (policy domains). By integrating research reporting standards into SALIENT, the framework is grounded in rigorous evaluation methodologies. The framework requires validation as being applicable to real-world studies of deployed AI models.

CONCLUSIONS

A novel end-to-end framework has been developed for implementing AI within hospital clinical practice that builds on previous AI implementation frameworks and research reporting standards.

Developing Consensus-Based Guidelines for Case Reporting in Aesthetic Medicine: Enhancing Transparency and Standardization

Clinical case reporting plays a vital role in sharing detailed patient narratives, providing insights into rare conditions, innovative treatments, and unexpected outcomes. However, existing reporting guidelines in aesthetic medicine fail to capture the specific nuances of procedures and outcomes in this field. The authors' objectives were to develop comprehensive guidelines for Case REporting in Aesthetic Medicine (CREAM). The study employed a 3-phase consensus process, including a literature review, expert interviews, and a consensus meeting. A diverse group of 10 expert participants (plastic surgeons, dermatologists, noncore specialists, evidence-based medicine expert, and research scientist) in Phase I and 30 experienced aesthetic practitioners in Phase II contributed to the research. Statistical analysis was conducted to assess agreement levels among participants and explore associations and variations within the data. The participants represented various specialties, genders, LGBTQ+ identities, and ethnic backgrounds. The research resulted in the development of the CREAM guidelines, consisting of a 16-item checklist. The guidelines covered essential aspects of case reporting, such as patient and practice information, procedure details, clinical assessment and outcomes, adverse events, and ethical considerations. Statistical analysis indicated a high level of consensus among participants, as well as significant associations between checklist items. CREAM guidelines represent a step toward enhancing transparency and standardization in case reporting in aesthetic medicine. Adhering to these guidelines will allow authors to contribute to a robust evidence base, prioritize patient safety, and drive advancements aesthetic medicine.

Perspectives on scientific error

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.

Overviews of reviews in the cardiovascular field underreported critical methodological and transparency characteristics: a methodological study based on the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement

OBJECTIVES

This study aimed to evaluate the epidemiology, reporting characteristics, and adherence to the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement of overviews of reviews (overviews) of interventions in the cardiovascular field.

STUDY DESIGN AND SETTING

MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched from January 1, 2000, to October 15, 2020. An updated search was performed in MEDLINE, Epistemonikos, and Google Scholar up to August 25, 2022. Overviews of interventions published in English and primarily considering populations, interventions, and outcomes pertinent to the cardiovascular field were eligible. Study selection, data extraction, and PRIOR adherence assessment were performed by two authors independently.

RESULTS

We analyzed 96 overviews. Almost half (43/96 [45%]) were published between 2020 and 2022 and contained a median of 15 systematic reviews (SRs) (interquartile range, 9-28). The commonest title terminology was "overview of (systematic) reviews" (38/96 [40%]). Methods for handling SR overlap were reported in 24/96 (25%), methods for assessing primary study overlap in 18/96 (19%), handling of discrepant data in 11/96 (11%), and methods for methodological quality or risk of bias assessment of the primary studies within SRs in 23/96 (24%). Authors included data sharing statements in 28/96 (29%), complete funding disclosure in 43/96 (45%), protocol registration in 43/96 (45%), and conflict of interest statement in 82/96 (85%) overviews.

CONCLUSION

Insufficient reporting was identified in methodological characteristics unique in overviews' conduct and most transparency markers. Adoption of PRIOR from the research community could ameliorate overviews' reporting.

Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review

OBJECTIVE

Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors.

METHODS

We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time.

RESULTS

187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles.

CONCLUSION

The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.

AgroEcoList 1.0: A checklist to improve reporting standards in ecological research in agriculture

Many publications lack sufficient background information (e.g. location) to be interpreted, replicated, or reused for synthesis. This impedes scientific progress and the application of science to practice. Reporting guidelines (e.g. checklists) improve reporting standards. They have been widely taken up in the medical sciences, but not in ecological and agricultural research. Here, we use a community-centred approach to develop a reporting checklist (AgroEcoList 1.0) through surveys and workshops with 23 experts and the wider agroecological community. To put AgroEcoList in context, we also assessed the agroecological community's perception of reporting standards in agroecology. A total of 345 researchers, reviewers, and editors, responded to our survey. Although only 32% of respondents had prior knowledge of reporting guidelines, 76% of those that had said guidelines improved reporting standards. Overall, respondents agreed on the need of AgroEcolist 1.0; only 24% of respondents had used reporting guidelines before, but 78% indicated they would use AgroEcoList 1.0. We updated AgroecoList 1.0 based on respondents' feedback and user-testing. AgroecoList 1.0 consists of 42 variables in seven groups: experimental/sampling set-up, study site, soil, livestock management, crop and grassland management, outputs, and finances. It is presented here, and is also available on github (https://github.com/AgroecoList/Agroecolist). AgroEcoList 1.0 can serve as a guide for authors, reviewers, and editors to improve reporting standards in agricultural ecology. Our community-centred approach is a replicable method that could be adapted to develop reporting checklists in other fields. Reporting guidelines such as AgroEcoList can improve reporting standards and therefore the application of research to practice, and we recommend that they are adopted more widely in agriculture and ecology.

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Protocol for a systematic review of reporting standards of lower limb endovascular interventions in peripheral arterial disease

INTRODUCTION

Techniques and devices for the endovascular treatment of peripheral arterial disease (PAD) are continuously evolving. High-quality clinical trials limit the variation in how endovascular interventions are described, performed and reported. The aim of this systematic review is to assess the quality of reporting standards in randomised controlled trials (RCTs) of endovascular lower limb interventions against the Consolidated Standards of Reporting Trials for Non-Pharmacologic Treatments (CONSORT-NPT) and template for intervention description and replication (TIDieR) framework.

METHODS

Randomised trials including participants with peripheral arterial disease undergoing any infra-inguinal lower limb endovascular arterial intervention, searched from Medline, Embase and Cochrane Library databases from inception to December 2021, will be included. All study data, including details of the procedure investigated, will be extracted in keeping with the CONSORT-NPT and TIDieR checklist. Descriptive statistics will be used to summarise general study details and reporting standards of the trials.

DISCUSSION

The results will be used to inform the design of future RCTs in this area by optimising the way the interventions are described, standardised, and monitored. The systematic review will be disseminated via peer-reviewed manuscripts and presentations at relevant conferences.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022288214.

'Small Data' for big insights in ecology

Big Data science has significantly furthered our understanding of complex systems by harnessing large volumes of data, generated at high velocity and in great variety. However, there is a risk that Big Data collection is prioritised to the detriment of 'Small Data' (data with few observations). This poses a particular risk to ecology where Small Data abounds. Machine learning experts are increasingly looking to Small Data to drive the next generation of innovation, leading to development in methods for Small Data such as transfer learning, knowledge graphs, and synthetic data. Meanwhile, meta-analysis and causal reasoning approaches are evolving to provide new insights from Small Data. These advances should add value to high-quality Small Data catalysing future insights for ecology.

Adherence to CONSORT Guidelines and Reporting of the Determinants of External Validity in Clinical Oncology Randomized Controlled Trials: A Review of Trials Published in Four Major Journals between 2013 and 2015

Our primary objective was to determine the proportion of trials that report the number of patients assessed for eligibility before randomization. We performed the systematic retrieval and analysis of all phase II, III, and IV RCTs published between 2013 and 2015 in four high-impact-factor journals in the field of clinical oncology. Among 456 RCTs reviewed, 236 trials (51.8%) reported the number of patients assessed for eligibility. Among the 236 trials that reported the entire enrollment process, the reasons for patient exclusion could be found in 184 trials (78%). A flow diagram was presented in 452 trials (99.1%), and 98 trials (21.5%) included a discussion on generalizability. Reporting the parameters of external validity in medical oncology RCTs is challenging. Improving adherence to the 2010 CONSORT guidelines concerning the enrollment process could help clinicians and health policymakers establish to whom trial results apply.

Reporting Eye-tracking Studies In DEntistry (RESIDE) checklist

OBJECTIVES

To (i) provide a scoping review of eye-tracking studies in dentistry, and (ii) propose a "Reporting Eye-tracking Studies in DEntistry" (RESIDE) checklist to facilitate standard reporting of eye-tracking studies.

DATA

A comprehensive search of six distinct electronic databases was undertaken.

SOURCES

Pubmed, OVID, Wed of Knowledge, Scopus, Cochrane and Google Scholar were used to identify studies that employed eye-tracking technology and dentistry as a subfield STUDY SELECTION: 42 studies met the inclusion criteria. Most studies exhibited several inconsistencies or failed to report on the appropriate items in the RESIDE checklist. These essential components include ethical approval, sample size calculation, location and setting, eye-tracking device attributes, participant calibration, sequence of events, and eye-tracking metrics (quantitative, qualitative and data details).

CONCLUSIONS

Evaluation of the published eye-tracking studies in this scoping review provides empirical data, highlighting the inconsistencies and limitations. Importantly, it illustrates the applicability of the RESIDE Checklist, which provides a comprehensive list of reporting elements to assist authors and reviewers of eye-tracking studies in dentistry. Also, RESIDE provides a framework to overcome critical issues to ensure high-quality scientific publications.

CLINICAL SIGNIFICANCE

A minimum threshold should be applied before accepting eye-tracking studies for publication in the future. RESIDE checklist promotes transparent and reproducible scientific communication about eye-tracking applications to dentistry. In addition, it provides a comprehensive list of reporting elements to assist authors and reviewers in ensuring high-quality scientific publications.

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

OBJECTIVE

Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records.

METHODS

We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies.

RESULTS

Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700.

CONCLUSION

In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.

The quality of reporting of randomized controlled trials of HuatuoZaizao pill for stroke

Background: HuatuoZaizao pill (HZP), a Chinese patent medicine, is often used in the treatment of stroke. However, there is still a lack of enough evidence to recommend the routine use of HZP for stroke. This study is aimed at evaluating the quality of reporting of randomized controlled trials (RCTs) on HZP for stroke. Methods: RCTs on HuatuoZaizao pill for stroke were evaluated by using Consolidated Standards of Reporting Trials (CONSORT) guidelines and CONSORT extension criteria on reporting herbal interventions (CONSORT-CHM) guidelines. Microsoft Excel 2007 and SPSS20.0 was used for statistics analyses. Results: Seventeen studies involving 1801 stroke patients were identified. CONSORT-CHM has expanded 24.3% (9/37) items in CONSORT and added a small item. The average scores of CONSORT evaluation is 14.6, while the average scores of the CONSORT-CHM evaluation is 11.6. The central items in CONSORT as eligibility criterion, sample size calculation, primary outcome, method of randomization sequence generation, allocation concealment, implementation of randomization, description of blinding, and detailed statistical methods were reported in 77%, 6%, 100%, 47%, 6%, 6%, 6%, and 94% of trials, respectively. In terms of the CONSORT-CHM, none of the articles reported in detail the dosage form, origin, formula basis and so on of HZP, and only half of studies reported the outcome indicators related to Traditional Chinese Medicine syndromes. Conclusion: The overall report quality of RCT related to HZP is low. HZP still needs to report higher quality RCTs to prove its effectiveness and safety.

Consistency of recommendations of clinical practice guidelines in periodontology: a systematic review

The aim of this systematic review was to evaluate the methodological quality and the consistency of recommendations of clinical practice guidelines (CPGs) in Periodontology. An electronic search was conducted in two databases, MEDLINE and EMBASE, eight CPGs databases, and home pages of scientific societies in Periodontology up to April 2022. Three reviewers independently assessed methodological quality using the AGREE II instrument. In addition, we evaluated the consistency of the recommendations. Eleven CPGs were included, and the topics developed focused on prevention, diagnosis, risk factors, surgical and non-surgical periodontal treatment, antimicrobial therapy, root coverage, and maintenance. We found that the AGREE domains 2 (Stakeholder involvement) and 5 (Applicability) obtained the lowest scores. Domains 1 (Scope and purpose), 3 (Rigor of development) and 4 (Clarity of presentation) obtained the highest scores among the evaluated CPGs. The clinical recommendations for treatment of periodontal diseases were mostly consistent. Overall, the quality of CPGs used in periodontics was high. There was consistency of recommendations in specific fields. These findings may help researchers to promote CPGs focused on different fields of periodontics that have not yet been developed. Furthermore, the clinician will be able to make better clinical decisions.

Development and psychometric evaluation of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool: a study protocol

INTRODUCTION

The need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research.

METHODS AND ANALYSIS

This study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published in Implementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field.

ETHICS AND DISSEMINATION

This study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King's College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).

Survey of quality and clarity of methods and results reporting in 1 year of intervention studies published in high-impact medical and psychiatric journals

OBJECTIVE

We assessed how well articles in major medical and psychiatric journals followed best reporting practices in presenting results of intervention studies.

METHOD

Standardised data collection was used to review studies in high-impact and widely read medical (JAMA, Lancet and New England Journal of Medicine) and psychiatric (American Journal of Psychiatry, JAMA Psychiatry, Journal of Clinical Psychiatry and Lancet Psychiatry) journals, published between 1 September 2018 and 31 August 2019. Two team members independently reviewed each article.

MEASURES

The primary outcome measure was proportion of papers reporting consensus elements required to understand and evaluate the results of the intervention. The secondary outcome measure was comparison of complete and accessible reporting in the major medical versus the major psychiatric journals.

RESULTS

One hundred twenty-seven articles were identified for inclusion. At least 90% of articles in both medical and psychiatric journals included sample size, statistical significance, randomisation method, elements of study flow, and age, sex, and illness severity by randomisation group. Selected elements less frequently reported by either journal type were confidence intervals in the abstract, reported in 93% (95% CI 84% to 97%) of medical journal articles and 58% (95% CI 45% to 69%) of psychiatric journal articles, and sample size method (93%, 95% CI 84% to 97% medical; 69%, 95% CI 57% to 80% psychiatric), race and ethnicity by randomisation group (51%, 95% CI 40% to 63% medical; 73%, 95% CI 60% to 83% psychiatric), and adverse events (94%; 95% CI 86% to 98% medical; 80%, 95% CI 68% to 88% psychiatric) in the main text. CIs were included less often in psychiatric than medical journals (p<0.004 abstract, p=0.04 main text, after multiple-testing correction).

CONCLUSIONS

Recommendations include standard inclusion of a table specifying the outcome(s) designated as primary, and the sample size, effect size(s), CI(s) and p value(s) corresponding to the primary test(s) for efficacy.

Feasibility of an Audit and Feedback Intervention to Facilitate Journal Policy Change Towards Greater Promotion of Transparency and Openness in Sports Science Research

Objectives

To evaluate (1) the feasibility of an audit-feedback intervention to facilitate sports science journal policy change, (2) the reliability of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) policy evaluation form, and (3) the extent to which policies of sports science journals support transparent and open research practices.

Methods

We conducted a cross-sectional, audit-feedback, feasibility study of transparency and openness standards of the top 38 sports science journals by impact factor. The TRUST form was used to evaluate journal policies support for transparent and open research practices. Feedback was provided to journal editors in the format of a tailored letter. Inter-rater reliability and agreement of the TRUST form was assessed using intraclass correlation coefficients and the standard error of measurement, respectively. Time-based criteria, fidelity of intervention delivery and qualitative feedback were used to determine feasibility.

Results

The audit-feedback intervention was feasible based on the time taken to rate journals and provide tailored feedback. The mean (SD) score on the TRUST form (range 0–27) was 2.05 (1.99), reflecting low engagement with transparent and open practices. Inter-rater reliability of the overall score of the TRUST form was moderate [ICC (2,1) = 0.68 (95% CI 0.55–0.79)], with standard error of measurement of 1.17. However, some individual items had poor reliability.

Conclusion

Policies of the top 38 sports science journals have potential for improved support for transparent and open research practices. The feasible audit-feedback intervention developed here warrants large-scale evaluation as a means to facilitate change in journal policies.

Registration: OSF (https://osf.io/d2t4s/).

Supplementary Information

The online version contains supplementary material available at 10.1186/s40798-022-00496-x.

Recommendations to improve use and reporting of statistics in animal experiments

Flaws in experimental statistics are a major contributor to the poor reproducibility of animal experiments. Informed decisions about whether conclusions are justified requires clear reporting of experimental data and the statistical methods used to analyse them. When data are misinterpreted, manipulated or concealed to generate publications, it creates an illusion that chance observations are robust data which confirm the hypotheses presented. Attempts to reproduce and advance such observations can propagate large areas of irreproducible science. This hinders scientific progress, erodes public support for research, damages reputations and wastes resources. This review analyses and explains recommendations to improve use and reporting of statistics in animal experiments.

[The Delphi technique: Methodology, variants and usage examples]

In the field of medicine and health sciences, Delphi methods are applied mainly in the exploratory or evaluative phases of a research process. Explicit and implicit knowledge of respected experts from research and practice is systematically synthesized. Originally developed as a method for structuring a group communication process, Delphi techniques have been established in the health sector as a consensus method. The findings are used to improve the evidence and acceptance of planned interventions or necessary standards or guidelines and to increase the probability of successful implementation in practice. However, different variants of Delphi methods have been developed in recent years, which are systematically contrasted and reflected in this paper with regard to key epistemological and methodological research activities. Based on this overview, researchers should be enabled to select the most suitable Delphi technique for their own research questions and research endeavors.

'Invisible actors'-How poor methodology reporting compromises mouse models of oncology: A cross-sectional survey

The laboratory mouse is a key player in preclinical oncology research. However, emphasis of techniques reporting at the expense of critical animal-related detail compromises research integrity, animal welfare, and, ultimately, the translation potential of mouse-based oncology models. To evaluate current reporting practices, we performed a cross-sectional survey of 400 preclinical oncology studies using mouse solid-tumour models. Articles published in 2020 were selected from 20 journals that specifically endorsed the ARRIVE (Animal Research: Reporting of In Vivo Experiments) preclinical reporting guidelines. We assessed reporting compliance for 22 items in five domains: ethical oversight assurance, animal signalment, husbandry, welfare, and euthanasia. Data were analysed using hierarchical generalised random-intercept models, clustered on journal. Overall, reporting of animal-related items was poor. Median compliance over all categories was 23%. There was little or no association between extent of reporting compliance and journal or journal impact factor. Age, sex, and source were reported most frequently, but verifiable strain information was reported for <10% of studies. Animal husbandry, housing environment, and welfare items were reported by <5% of studies. Fewer than one in four studies reported analgesia use, humane endpoints, or an identifiable method of euthanasia. Of concern was the poor documentation of ethical oversight information. Fewer than one in four provided verifiable approval information, and almost one in ten reported no information, or information that was demonstrably false. Mice are the "invisible actors" in preclinical oncology research. In spite of widespread endorsement of reporting guidelines, adherence to reporting guidelines on the part of authors is poor and journals fail to enforce guideline reporting standards. In particular, the inadequate reporting of key animal-related items severely restricts the utility and translation potential of mouse models, and results in research waste. Both investigators and journals have the ethical responsibility to ensure animals are not wasted in uninformative research.

Effect of a Run-In Period on Estimated Treatment Effects in Cardiovascular Randomized Clinical Trials: A Meta-Analytic Review

Background A run-in period may increase adherence to intervention and reduce loss to follow-up. Whether use of a run-in period affects the magnitude of treatment effects is unknown. Methods and Results We conducted a meta-analysis comparing treatment effects from 11 systematic reviews of cardiovascular prevention trials using a run-in period with matched trials not using a run-in period. We matched run-in with non-run-in trials by population, intervention, control, and outcome. We calculated a ratio of relative risks (RRRs) using a random-effects meta-analysis. Our primary outcome was a composite of cardiovascular events, and the primary analysis was a matched comparison of clinical trials using a run-in period versus without a run-in period. We identified 66 run-in trials and 111 non-run-in trials (n=668 901). On meta-analysis there was no statistically significant difference in the magnitude of treatment effect between run-in trials (relative risk [RR], 0.83 [95% CI, 0.80-0.87]) compared with non-run-in trials (RR, 0.88 [95% CI, 0.84-0.91]; RRR, 0.95 [95% CI, 0.90-1.01]). There was no significant difference in the RRR for secondary outcomes of all-cause mortality (RRR, 0.97 [95% CI, 0.91-1.03]) or medication discontinuation because of adverse events (RRR, 1.05 [95% CI, 0.85-1.21]). Post hoc exploratory univariate meta-regression showed that on average a run-in period is associated with a statistically significant difference in treatment effect (RRR, 0.94 [95% CI, 0.90-0.99]) for cardiovascular composite outcome, but this was not statistically significant on multivariable meta-regression analysis (RRR, 0.95 [95% CI, 0.90-1.0]). Conclusions The use of a run-in period was not associated with a difference in the magnitude of treatment effect among cardiovascular prevention trials.

Methodological and ethical quality of surgical trials from 2016 to 2020

PURPOSE

Randomized controlled trials (RCTs) are the gold standard tool used to evaluate therapeutic interventions. The published results showed that progress still needs to be made not only from a methodological point of view but also from an ethical point of view. The aim of this study was to evaluate the methodological and ethical qualities of randomized clinical trials in surgery over the last few years.

METHODS

All of the articles chosen for review reported on randomized controlled surgical trials and were published in 10 international journals between 2016 and 2020. Eligible studies were identified, selected, and then evaluated based on a broad set of predetermined criteria. Methodological quality was evaluated using the Jadad scale, and ethical quality was evaluated using the Berdeu score.

RESULTS

The methodological quality score (Jadad scale) ranged from 5 to 13, with a mean of 10.0 ± 1.54. The methodological quality was insufficient (score ≤ 9) for 162 trials (31.2%). The ethical quality score ranged from 0.25 to 1, with a mean of 0.8 ± 0.11. Fifty-two articles (10%) did not state that informed consent was requested from the participants, and 21 articles (4%) did not report either research ethics committee or institutional committee protocol approval.

CONCLUSION

The randomized clinical surgical trials analyzed showed that they had satisfactory methods in only 70% of the cases and that they had respected the fundamental ethical principles in 90% of the cases. However, less than 8% of the studies reported planned interim analysis, prospectively defined stopping rules, and independent monitoring committee.

Consensus Head Acceleration Measurement Practices (CHAMP): Origins, Methods, Transparency and Disclosure

The use of head kinematic measurement devices has recently proliferated owing to technology advances that make such measurement more feasible. In parallel, demand to understand the biomechanics of head impacts and injury in sports and the military has increased as the burden of such loading on the brain has received focused attention. As a result, the field has matured to the point of needing methodological guidelines to improve the rigor and consistency of research and reduce the risk of scientific bias. To this end, a diverse group of scientists undertook a comprehensive effort to define current best practices in head kinematic measurement, culminating in a series of manuscripts outlining consensus methodologies and companion summary statements. Summary statements were discussed, revised, and voted upon at the Consensus Head Acceleration Measurement Practices (CHAMP) Conference in March 2022. This manuscript summarizes the motivation and methods of the consensus process and introduces recommended reporting checklists to be used to increase transparency and rigor of future experimental design and publication of work in this field. The checklists provide an accessible means for researchers to apply the best practices summarized in the companion manuscripts when reporting studies utilizing head kinematic measurement in sport and military settings.