1
|
Kocak B, Akinci D'Antonoli T, Ates Kus E, Keles A, Kala A, Kose F, Kadioglu M, Solak S, Sunman S, Temiz ZH. Self-reported checklists and quality scoring tools in radiomics: a meta-research. Eur Radiol 2024; 34:5028-5040. [PMID: 38180530 DOI: 10.1007/s00330-023-10487-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 11/11/2023] [Accepted: 11/24/2023] [Indexed: 01/06/2024]
Abstract
OBJECTIVE To evaluate the use of reporting checklists and quality scoring tools for self-reporting purposes in radiomics literature. METHODS Literature search was conducted in PubMed (date, April 23, 2023). The radiomics literature was sampled at random after a sample size calculation with a priori power analysis. A systematic assessment for self-reporting, including the use of documentation such as completed checklists or quality scoring tools, was conducted in original research papers. These eligible papers underwent independent evaluation by a panel of nine readers, with three readers assigned to each paper. Automatic annotation was used to assist in this process. Then, a detailed item-by-item confirmation analysis was carried out on papers with checklist documentation, with independent evaluation of two readers. RESULTS The sample size calculation yielded 117 papers. Most of the included papers were retrospective (94%; 110/117), single-center (68%; 80/117), based on their private data (89%; 104/117), and lacked external validation (79%; 93/117). Only seven papers (6%) had at least one self-reported document (Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), or Checklist for Artificial Intelligence in Medical Imaging (CLAIM)), with a statistically significant binomial test (p < 0.001). Median rate of confirmed items for all three documents was 81% (interquartile range, 6). For quality scoring tools, documented scores were higher than suggested scores, with a mean difference of - 7.2 (standard deviation, 6.8). CONCLUSION Radiomic publications often lack self-reported checklists or quality scoring tools. Even when such documents are provided, it is essential to be cautious, as the accuracy of the reported items or scores may be questionable. CLINICAL RELEVANCE STATEMENT Current state of radiomic literature reveals a notable absence of self-reporting with documentation and inaccurate reporting practices. This critical observation may serve as a catalyst for motivating the radiomics community to adopt and utilize such tools appropriately, thereby fostering rigor, transparency, and reproducibility of their research, moving the field forward. KEY POINTS • In radiomics literature, there has been a notable absence of self-reporting with documentation. • Even if such documents are provided, it is critical to exercise caution because the accuracy of the reported items or scores may be questionable. • Radiomics community needs to be motivated to adopt and appropriately utilize the reporting checklists and quality scoring tools.
Collapse
Affiliation(s)
- Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey.
| | - Tugba Akinci D'Antonoli
- Institute of Radiology and Nuclear Medicine, Cantonal Hospital Baselland, Liestal, Switzerland
| | - Ece Ates Kus
- Department of Neuroradiology, Klinikum Lippe, Lemgo, Germany
| | - Ali Keles
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Ahmet Kala
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Fadime Kose
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Mehmet Kadioglu
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Sila Solak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Seyma Sunman
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| | - Zisan Hayriye Temiz
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey
| |
Collapse
|
2
|
Bała MM, Poklepović Peričić T, Žuljević MF, Bralić N, Zając J, Motaze NV, Rohwer A, Gajdzica M, Young T. Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study. BMJ Evid Based Med 2024; 29:229-238. [PMID: 38862202 DOI: 10.1136/bmjebm-2023-112647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/28/2024] [Indexed: 06/13/2024]
Abstract
OBJECTIVES The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting. DESIGN Methodological cross-sectional study. METHODS Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively. RESULTS We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037). CONCLUSIONS Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result. STUDY REGISTRATION NUMBER The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).
Collapse
Affiliation(s)
- Małgorzata M Bała
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Tina Poklepović Peričić
- Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
| | - Marija Franka Žuljević
- Department of Medical Humanities, University of Split School of Medicine, Split, Croatia
| | - Nensi Bralić
- Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
| | - Joanna Zając
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Nkengafac Villyen Motaze
- Medicine Usage in South Africa, Faculty of Health Sciences, North-West University, Potchefstroom, South Africa
| | - Anke Rohwer
- Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University Faculty of Medicine and Health Sciences, Cape Town, Western Cape, South Africa
| | - Michalina Gajdzica
- Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Taryn Young
- Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University Faculty of Medicine and Health Sciences, Cape Town, Western Cape, South Africa
| |
Collapse
|
3
|
Best AM, Lang TA, Greenberg BL, Gunsolley JC, Ioannidou E. The Oral Health Statistics Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration. J Oral Maxillofac Surg 2024:S0278-2391(24)00584-6. [PMID: 39032518 DOI: 10.1016/j.joms.2024.06.174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2024] [Accepted: 06/24/2024] [Indexed: 07/23/2024]
Abstract
Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the Oral Health Statistics Guidelines. Drafts were circulated to the editors of 85 oral health journals and to task force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the task force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.
Collapse
Affiliation(s)
- Al M Best
- Professor Emeritus, School of Dentistry and Department of Biostatistics, School of Medicine, Virginia Commonwealth University, Richmond, VA
| | - Thomas A Lang
- Adjunct Faculty, University of Chicago Medical Writing Program
| | - Barbara L Greenberg
- Adjunct Professor, Epidemiology and Biostatistics, Touro College of Dental Medicine at New York Medical College, Valhalla, NY
| | - John C Gunsolley
- Professor Emeritus, School of Dentistry, Virginia Commonwealth University, Richmond, VA
| | - E Ioannidou
- Professor and Chair of Orofacial Sciences, UCSF School of Dentistry, San Francisco, CA.
| |
Collapse
|
4
|
Mahmoudian A, Ashouri A, Bilandi RR, Mohammadzadeh F, Dashti S, Bahri N. The possible short-term of Nigella sativa - L in the management of adolescent polycystic ovarian syndrome: results of a randomized controlled trial. J Ovarian Res 2024; 17:144. [PMID: 38997723 PMCID: PMC11242022 DOI: 10.1186/s13048-024-01460-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2024] [Accepted: 06/18/2024] [Indexed: 07/14/2024] Open
Abstract
BACKGROUND Polycystic ovary syndrome (PCOS) is the most common endocrine disorder in reproductive age and the most common cause of infertility due to anovulation. PCOS in adolescents is concerning. Nigella sativa is effective in improving gonadotropins and sex hormones. The current study was designed to investigate the effect of Nigella sativa supplementation on PCOS symptoms and their severity in adolescents. METHODS The current randomized clinical trial was conducted on 114 adolescents with PCOS who were referred to gynecologist offices and clinics in Gonabad, Iran from March 2022 to March 2023. Participants were randomly allocated to the intervention (Nigella sativa 1000 mg/day) and control (10 mg/day medroxyprogesterone from the 14th day of the cycle for 10 nights) groups. The study duration was 16 weeks. Ovarian volume (measured by ultrasound), anthropometric and blood pressure; serum testosterone, dehydroepiandrosterone (DHEA), dehydroepiandrosterone sulfate (DHEA-S), luteinizing hormone (LH), hirsutism severity (Ferriman-Gallwey score) levels were evaluated before and after the study. RESULTS Data from 103 participants (control group = 53, intervention group = 50) were analyzed. The mean age of participants was 17.0 (Interquartile range [IQR]:2.0). The mean difference in hirsutism score changes (p < 0.001), right (p = 0.002), and left (p = 0.010) ovarian volume, serum LH (p < 0.001) and testosterone (p = 0.001) were significantly higher in the intervention group compared to the control group. The frequency of oligomenorrhea, menometrorrhagia, and amenorrhea, were significantly reduced after the study in the intervention group compared to the control group (ps < 0.001). CONCLUSIONS Short-term Nigella sativa supplementation may be effective in reducing ovarian volume and improving hormonal balance, and menstrual irregularities in adolescents with PCOS. Further research and long-term studies are warranted to validate the potential therapeutic effects of Nigella sativa in adolescents with PCOS. IRCT REGISTRATION NUMBER IRCT20221017056209N1 Registration date: 2022-11-22.
Collapse
Affiliation(s)
- Azamsadat Mahmoudian
- Department of Obstetrics & Gynecology, School of Medicine, Social Determinants of Health Research Center, Gonabad University of Medical Sciences, Gonabad, Iran
| | - Akram Ashouri
- Department of Midwifery, School of Medicine, Social Determinants of Health Research Center, Gonabad University of Medical Sciences, Gonabad, Iran
| | - Roghaieh Rahmani Bilandi
- Department of Midwifery, Faculty of Medicine, Reproductive Health and Population Research Center, Gonabad University of Medical Sciences, Gonabad, Iran
| | - Fatemeh Mohammadzadeh
- Department of Epidemiology and Biostatistics, School of Health, Gonabad University of Medical Sciences, Gonabad, Iran
| | - Sareh Dashti
- Department of Midwifery, Faculty of Nursing and Midwifery, Mashhad Medical Sciences, Islamic Azad University, Mashhad, Iran
- Department of Public Health, Faculty of Paramedicine, Mashhad Medical Sciences, Islamic Azad University, Mashhad, Iran
| | - Narjes Bahri
- Department of Midwifery, School of Medicine, Social Determinants of Health Research Center, Gonabad University of Medical Sciences, Gonabad, Iran.
| |
Collapse
|
5
|
Best AM, Lang TA, Greenberg BL, Gunsolley JC, Ioannidou E. The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: explanation and elaboration. J Am Dent Assoc 2024:S0002-8177(24)00316-7. [PMID: 39001723 DOI: 10.1016/j.adaj.2024.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/15/2024]
Abstract
Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.
Collapse
|
6
|
Best AM, Lang TA, Greenberg BL, Gunsolley JC, Ioannidou E. The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration. JDR Clin Trans Res 2024:23800844241247029. [PMID: 38993046 DOI: 10.1177/23800844241247029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/13/2024] Open
Abstract
Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.
Collapse
Affiliation(s)
- A M Best
- School of Dentistry and Department of Biostatistics, School of Medicine, Virginia Commonwealth University, Richmond, VA, USA
| | - T A Lang
- University of Chicago Medical Writing Program, Chicago, IL, USA
| | - B L Greenberg
- Epidemiology and Biostatistics, Touro College of Dental Medicine at New York Medical College, Valhalla, NY, USA
| | - J C Gunsolley
- School of Dentistry, Virginia Commonwealth University, Richmond, VA, USA
| | - E Ioannidou
- UCSF School of Dentistry, San Francisco, CA, USA
| |
Collapse
|
7
|
Ben-Porat T, Yousefi R, Stojanovic J, Boucher VG, Fortin A, Lavoie K, Bacon SL. Completeness of intervention reporting of nutrition-focused weight management interventions adjunct to metabolic and bariatric surgery: effect of the TIDieR checklist. Int J Obes (Lond) 2024; 48:901-912. [PMID: 38459257 DOI: 10.1038/s41366-024-01506-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 02/20/2024] [Accepted: 02/26/2024] [Indexed: 03/10/2024]
Abstract
Nutrition-focused interventions are essential to optimize the bariatric care process and improve health and weight outcomes over time. Clear and detailed reporting of these interventions in research reports is crucial for understanding and applying the findings effectively in clinical practice and research replication. Given the importance of reporting transparency in research, this study aimed to use the Template for Intervention Description and Replication (TIDieR) checklist to evaluate the completeness of intervention reporting in nutritional weight management interventions adjunct to metabolic and bariatric surgery (MBS). The secondary aim was to examine the factors associated with better reporting. A literature search in PubMed, PsychINFO, EMBASE, Scopus, and the Cochrane Controlled Register of Trials was conducted to include randomized controlled trials (RCT), quasi-RCTs and parallel group trials. A total of 22 trials were included in the final analysis. Among the TIDieR 12 items, 6.6 ± 1.9 items were fully reported by all studies. None of the studies completely reported all intervention descriptors. The main areas where reporting required improvement were providing adequate details of the materials and procedures of the interventions, intervention personalization, and intervention modifications during the study. The quality of intervention reporting remained the same after vs. before the release of the TIDieR guidelines. Receiving funds from industrial organizations (p = 0.02) and having the study recorded within a registry platform (p = 0.08) were associated with better intervention reporting. Nutritional weight management interventions in MBS care are still below the desirable standards for reporting. The present study highlights the need to improve adequate reporting of such interventions, which would allow for greater replicability, evaluation through evidence synthesis studies, and transferability into clinical practice.
Collapse
Affiliation(s)
- Tair Ben-Porat
- Montreal Behavioural Medicine Centre (MBMC), Centre intégré universitaire de santé et de services sociaux du Nord-de-l'Île-de-Montréal (CIUSSS-NIM), Montreal, Canada
- Department of Health, Kinesiology and Applied Physiology, Concordia University, Montreal, Canada
| | - Reyhaneh Yousefi
- Montreal Behavioural Medicine Centre (MBMC), Centre intégré universitaire de santé et de services sociaux du Nord-de-l'Île-de-Montréal (CIUSSS-NIM), Montreal, Canada
- Department of Health, Kinesiology and Applied Physiology, Concordia University, Montreal, Canada
| | - Jovana Stojanovic
- Canadian Agency for Drugs and Health Technologies in Health (CADTH), Ottawa, Canada
| | - Vincent Gosselin Boucher
- School of Kinesiology, Faculty of Education, The University of British Columbia, Vancouver, Canada
| | - Annabelle Fortin
- Montreal Behavioural Medicine Centre (MBMC), Centre intégré universitaire de santé et de services sociaux du Nord-de-l'Île-de-Montréal (CIUSSS-NIM), Montreal, Canada
- Department of Psychology, Université du Québec à Montréal, Montreal, Canada
| | - Kim Lavoie
- Montreal Behavioural Medicine Centre (MBMC), Centre intégré universitaire de santé et de services sociaux du Nord-de-l'Île-de-Montréal (CIUSSS-NIM), Montreal, Canada
- Department of Psychology, Université du Québec à Montréal, Montreal, Canada
| | - Simon L Bacon
- Montreal Behavioural Medicine Centre (MBMC), Centre intégré universitaire de santé et de services sociaux du Nord-de-l'Île-de-Montréal (CIUSSS-NIM), Montreal, Canada.
- Department of Health, Kinesiology and Applied Physiology, Concordia University, Montreal, Canada.
| |
Collapse
|
8
|
Fusaroli M, Salvo F, Begaud B, AlShammari TM, Bate A, Battini V, Brueckner A, Candore G, Carnovale C, Crisafulli S, Cutroneo PM, Dolladille C, Drici MD, Faillie JL, Goldman A, Hauben M, Herdeiro MT, Mahaux O, Manlik K, Montastruc F, Noguchi Y, Norén GN, Noseda R, Onakpoya IJ, Pariente A, Poluzzi E, Salem M, Sartori D, Trinh NTH, Tuccori M, van Hunsel F, van Puijenbroek E, Raschi E, Khouri C. The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement. Drug Saf 2024; 47:575-584. [PMID: 38713346 PMCID: PMC11116242 DOI: 10.1007/s40264-024-01421-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/07/2024] [Indexed: 05/08/2024]
Abstract
BACKGROUND AND AIM Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts. METHODS We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting. RESULTS Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts. CONCLUSIONS The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.
Collapse
Affiliation(s)
- Michele Fusaroli
- Department of Medical and Surgical Sciences, Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | - Francesco Salvo
- Université de Bordeaux, INSERM, BPH, Team AHeaD, U1219, 33000, Bordeaux, France
- Service de Pharmacologie Médicale, CHU de Bordeaux, INSERM, U1219, 33000, Bordeaux, France
| | - Bernard Begaud
- Université de Bordeaux, INSERM, BPH, Team AHeaD, U1219, 33000, Bordeaux, France
| | | | - Andrew Bate
- Global Safety, GSK, Brentford, UK
- Department of Non-Communicable Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
| | - Vera Battini
- Pharmacovigilance and Clinical Research, International Centre for Pesticides and Health Risk Prevention, Department of Biomedical and Clinical Sciences (DIBIC), ASST Fatebenefratelli-Sacco University Hospital, Università degli Studi di Milano, Milan, Italy
| | | | | | - Carla Carnovale
- Pharmacovigilance and Clinical Research, International Centre for Pesticides and Health Risk Prevention, Department of Biomedical and Clinical Sciences (DIBIC), ASST Fatebenefratelli-Sacco University Hospital, Università degli Studi di Milano, Milan, Italy
| | | | - Paola Maria Cutroneo
- Unit of Clinical Pharmacology, Sicily Pharmacovigilance Regional Centre, University Hospital of Messina, Messina, Italy
| | - Charles Dolladille
- UNICAEN, EA4650 SEILIRM, CHU de Caen Normandie, Normandie University, Caen, France
- Department of Pharmacology, CHU de Caen Normandie, Caen, France
| | - Milou-Daniel Drici
- Department of Clinical Pharmacology, Université Côte d'Azur Medical Center, Nice, France
| | - Jean-Luc Faillie
- Desbrest Institute of Epidemiology and Public Health, Department of Medical Pharmacology and Toxicology, INSERM, Univ Montpellier, Regional Pharmacovigilance Centre, CHU Montpellier, Montpellier, France
| | - Adam Goldman
- Department of Internal Medicine, Sheba Medical Center, Ramat-Gan, Israel
- Department of Epidemiology and Preventive Medicine, School of Public Health, Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Manfred Hauben
- Pfizer Inc., New York, USA
- Department of Family and Community Medicine, New York Medical College, Valhalla, New York, USA
| | - Maria Teresa Herdeiro
- Department of Medical Sciences, IBIMED-Institute of Biomedicine, University of Aveiro, 3810-193, Aveiro, Portugal
| | | | - Katrin Manlik
- Bayer AG, Medical Affairs and Pharmacovigilance, Berlin, Germany
| | - François Montastruc
- Department of Medical and Clinical Pharmacology, Centre of PharmacoVigilance and Pharmacoepidemiology, Faculty of Medicine, Toulouse University Hospital (CHU), Toulouse, France
- CIC 1436, Team PEPSS (Pharmacologie En Population cohorteS et biobanqueS), Toulouse University Hospital, Toulouse, France
| | - Yoshihiro Noguchi
- Laboratory of Clinical Pharmacy, Gifu Pharmaceutical University, Gifu, Japan
| | | | - Roberta Noseda
- Institute of Pharmacological Sciences of Southern Switzerland, Division of Clinical Pharmacology and Toxicology, Ente Ospedaliero Cantonale, Lugano, Switzerland
| | - Igho J Onakpoya
- Department for Continuing Education, University of Oxford, Oxford, UK
| | - Antoine Pariente
- Université de Bordeaux, INSERM, BPH, Team AHeaD, U1219, 33000, Bordeaux, France
- Service de Pharmacologie Médicale, CHU de Bordeaux, INSERM, U1219, 33000, Bordeaux, France
| | - Elisabetta Poluzzi
- Department of Medical and Surgical Sciences, Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | | | - Daniele Sartori
- Uppsala Monitoring Centre, Uppsala, Sweden
- Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Nhung T H Trinh
- PharmacoEpidemiology and Drug Safety Research Group, Department of Pharmacy, University of Oslo, Oslo, Norway
| | - Marco Tuccori
- Tuscany Regional Centre, Unit of Adverse Drug Reaction Monitoring, University Hospital of Pisa, Pisa, Italy
| | - Florence van Hunsel
- Netherlands Pharmacovigilance Centre Lareb, 's-Hertogenbosch, the Netherlands
- University of Groningen, Groningen Research Institute of Pharmacy, PharmacoTherapy, Epidemiology and Economics, Groningen, the Netherlands
| | - Eugène van Puijenbroek
- Netherlands Pharmacovigilance Centre Lareb, 's-Hertogenbosch, the Netherlands
- University of Groningen, Groningen Research Institute of Pharmacy, PharmacoTherapy, Epidemiology and Economics, Groningen, the Netherlands
| | - Emanuel Raschi
- Department of Medical and Surgical Sciences, Alma Mater Studiorum, University of Bologna, Bologna, Italy.
| | - Charles Khouri
- Pharmacovigilance Department, Univ. Grenoble Alpes, Grenoble Alpes University Hospital, Grenoble, France.
- UMR 1300-HP2 Laboratory, Univ. Grenoble Alpes, INSERM, Grenoble Alpes University, Grenoble, France.
| |
Collapse
|
9
|
Kapoor S, Cantrell EM, Peng K, Pham TH, Bail CA, Gundersen OE, Hofman JM, Hullman J, Lones MA, Malik MM, Nanayakkara P, Poldrack RA, Raji ID, Roberts M, Salganik MJ, Serra-Garcia M, Stewart BM, Vandewiele G, Narayanan A. REFORMS: Consensus-based Recommendations for Machine-learning-based Science. SCIENCE ADVANCES 2024; 10:eadk3452. [PMID: 38691601 PMCID: PMC11092361 DOI: 10.1126/sciadv.adk3452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 03/29/2024] [Indexed: 05/03/2024]
Abstract
Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and generalizability. These failures can hinder scientific progress, lead to false consensus around invalid claims, and undermine the credibility of ML-based science. ML methods are often applied and fail in similar ways across disciplines. Motivated by this observation, our goal is to provide clear recommendations for conducting and reporting ML-based science. Drawing from an extensive review of past literature, we present the REFORMS checklist (recommendations for machine-learning-based science). It consists of 32 questions and a paired set of guidelines. REFORMS was developed on the basis of a consensus of 19 researchers across computer science, data science, mathematics, social sciences, and biomedical sciences. REFORMS can serve as a resource for researchers when designing and implementing a study, for referees when reviewing papers, and for journals when enforcing standards for transparency and reproducibility.
Collapse
Affiliation(s)
- Sayash Kapoor
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| | - Emily M. Cantrell
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- School of Public and International Affairs, Princeton University, Princeton, NJ 08544, USA
| | - Kenny Peng
- Department of Computer Science, Cornell University, Ithaca, NY 14850, USA
| | - Thanh Hien Pham
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| | - Christopher A. Bail
- Department of Sociology, Duke University, Durham, NC 27708, USA
- Department of Political Science, Duke University, Durham, NC 27708, USA
- Sanford School of Public Policy, Duke University, Durham, NC 27708, USA
| | - Odd Erik Gundersen
- Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway
- Aneo AS, Trondheim, Norway
| | | | - Jessica Hullman
- Department of Computer Science, Northwestern University, Evanston, IL 60208, USA
| | - Michael A. Lones
- School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK
| | - Momin M. Malik
- Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA
- School of Social Policy & Practice, University of Pennsylvania, Philadelphia, PA 19104, USA
- Institute in Critical Quantitative, Computational, & Mixed Methodologies, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Priyanka Nanayakkara
- Department of Computer Science, Northwestern University, Evanston, IL 60208, USA
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| | | | - Inioluwa Deborah Raji
- Department of Computer Science, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Michael Roberts
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK
- Department of Medicine, University of Cambridge, Cambridge, UK
| | - Matthew J. Salganik
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- Office of Population Research, Princeton University, Princeton, NJ 08544, USA
| | - Marta Serra-Garcia
- Rady School of Management, University of California, San Diego, La Jolla, CA 92093, USA
| | - Brandon M. Stewart
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- Office of Population Research, Princeton University, Princeton, NJ 08544, USA
- Department of Politics, Princeton University, Princeton, NJ 08544, USA
| | - Gilles Vandewiele
- Department of Information Technology, Ghent University, Ghent, Belgium
| | - Arvind Narayanan
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| |
Collapse
|
10
|
Jelinek T, Shumard A, Modi J, Smith C, Nees D, Hughes G, Vassar M. Endorsement of reporting guidelines and clinical trial registration across Scopus-indexed rheumatology journals: a cross-sectional analysis. Rheumatol Int 2024; 44:909-917. [PMID: 37861727 DOI: 10.1007/s00296-023-05474-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 09/16/2023] [Indexed: 10/21/2023]
Abstract
The purpose of this study was to investigate the instructions for authors of rheumatology journals and analyze their endorsement of reporting guidelines and clinical trial registration. Sixty rheumatology journals were selected by a research librarian and an investigator through the 2021 Scopus CiteScore tool. The instructions for authors' subsection of each journal was assessed to determine endorsement of study design-specific reporting guidelines or clinical trial registration. Descriptive statistics were calculated using R (version 4.2.1) and RStudio. Of the 58 journals analyzed, 34 (34/58; 59%) mentioned the EQUATOR Network: an online compendium of best practice reporting guidelines. The most commonly mentioned reporting guidelines were CONSORT with 44 journals (44/58; 75%), and PRISMA with 35 journals (35/58; 60%). The least mentioned guidelines were QUOROM with 56 journals not mentioning the guideline (56/58; 97%), and SRQR with 53 journals not mentioning the guideline (53/57, 93%). Clinical trial registration was required by 38 journals (38/58; 66%) and recommended by 8 journals (8/58; 14%). Our study found that endorsement of reporting guidelines and clinical trial registration within rheumatology journals was suboptimal with great room for improvement. Endorsement of reporting guidelines have shown to not only mitigate bias, but also improve research methodologies. Therefore, we recommend rheumatology journals broadly expand their endorsement of reporting guidelines and clinical trial registration to improve the quality of evidence they publish.
Collapse
Affiliation(s)
- Trevon Jelinek
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA.
| | - Alexandra Shumard
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA
| | - Jay Modi
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA
| | - Caleb Smith
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA
| | - Danya Nees
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA
| | | | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17Th St., Tulsa, OK, 74107, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| |
Collapse
|
11
|
Jimenez-Gomez C. Research and mentorship in behavior analysis from a lens of cultural responsiveness and antiracism. J Exp Anal Behav 2024; 121:373-388. [PMID: 38587283 DOI: 10.1002/jeab.911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Accepted: 03/07/2024] [Indexed: 04/09/2024]
Abstract
Although scientific endeavors strive to be objective, they are the work of individuals whose unique perspectives and experiences influence their research and interpretations of the world and data. Much has been said and written lately about the need to embed cultural responsiveness in behavior analysis and the need to enhance diversity in the field. In fact, similar conversations are taking place in many areas of science. Despite the current buzz, many behavioral researchers may be left wondering what they can do or whether it is incumbent on them to act. For the field of behavior analysis to move toward adopting the values of diversity, equity, inclusion, and access, members of the scientific community must actively engage in behaviors that foster inclusive and safe learning environments for students, engage in collaborative work, and incorporate culturally responsive research and mentorship practices. This article will describe some current practices, showcase exemplars of culturally responsive research and mentorship, and provide resources for researchers and mentors.
Collapse
|
12
|
Sen Yavuz B, Kaya R, Kodaman Dokumacigil N, Ozgur EG, Bekiroglu N, Kargul B. Clinical performance of short fiber reinforced composite and glass hybrid on hypomineralized molars: A 36-month randomized split-mouth study. J Dent 2024; 144:104919. [PMID: 38431187 DOI: 10.1016/j.jdent.2024.104919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Revised: 02/19/2024] [Accepted: 02/29/2024] [Indexed: 03/05/2024] Open
Abstract
OBJECTIVES This study aimed to assess the clinical performance outcome at 36 months of molars with molar incisor hypomineralization and carious lesions, treated with two different restorative approaches following selective caries removal. METHODS The children aged 6 to 12 years (18 female, 13 male) included in the study had at least two carious permanent first molars diagnosed with molar incisor hypomineralization. Sixty-two molars were restored in a split-mouth design. In all subjects, selective caries removal was performed so that caries was completely removed from the cavosurface walls and only soft dentin was left above the pulp chamber. Short fiber reinforced composite (SFRC; EverX Flow™) covered by micro-hybrid composite (G-Aenial® posterior composite) and Glass Hybrid (GH; Equia Forte® HT) were used as restorative materials. The restorations were evaluated according to modified United States Public Health Service (USPHS) criteria at baseline, 6, 12, 18, 24, and 36 month follow-ups. RESULTS During the 36-month follow-up, eight GH and four SFRC restorations failed. The clinical success of both restorations decreased statistically over time (p < 0.001 for both). When variables such as restoration type, sex, age, tooth type, and time were included in the model, the risk of failure of the restorations of the left lower first molar was statistically significantly higher than that of the left upper first molar (p < 0.002). CONCLUSION Direct composite restorations with SFRC and GH restorations perform similar clinical success with selective caries removal in the management of permanent molars affected by molar incisor hypomineralization. CLINICAL SIGNIFICANCE SFRC or GH restorations with similar clinical success might be preferred for the management of MIH-affected molars.
Collapse
Affiliation(s)
- Betul Sen Yavuz
- Department of Pediatric Dentistry, Faculty of Dentistry, Marmara University, Basibuyuk yolu, 9/3, Maltepe, Istanbul 34854, Turkey.
| | | | - Nur Kodaman Dokumacigil
- Department of Pediatric Dentistry, Faculty of Dentistry, Marmara University, Basibuyuk yolu, 9/3, Maltepe, Istanbul 34854, Turkey
| | - Emrah Gokay Ozgur
- Department of Biostatistics, Faculty of Medicine, Marmara University, Istanbul, Turkey
| | - Nural Bekiroglu
- Department of Biostatistics, Faculty of Medicine, Marmara University, Istanbul, Turkey
| | - Betul Kargul
- Department of Pediatric Dentistry, Faculty of Dentistry, Marmara University, Basibuyuk yolu, 9/3, Maltepe, Istanbul 34854, Turkey
| |
Collapse
|
13
|
Park MN, Kim SE, Choi S, Chang Y, Kim H, Lee HE, Lee SK, Sung MK, Paik HY. Sex reporting of cells used in cancer research: A systematic review. FASEB J 2024; 38:e23552. [PMID: 38498336 DOI: 10.1096/fj.202301986r] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 02/10/2024] [Accepted: 02/27/2024] [Indexed: 03/20/2024]
Abstract
Sex and gender disparities in biomedical research have been emphasized to improve scientific knowledge applied for the health of both men and women. Despite sex differences in cancer incidence, prognosis, and responses to therapeutic agents, mechanistic explanations at molecular levels are far from enough. Recent studies suggested that cell sex is an important biological variable due to differences in sex chromosome gene expression and differences in events associated with developmental biology. The objective of this study was to analyze the reporting of sex of cells used in cancer research using articles published in Cancer Cell, Molecular Cancer, Journal of Hematology & Oncology, Journal for ImmunoTherapy of Cancer, and Cancer Research in 2020, and to examine whether there exists any sex bias. We found that the percentage of cells with sex notation in the article was 36.5%. Primary cells exhibited higher sex notation compared to cell lines. A higher percentage of female cells were used in cell cultures with sex notation. Also, sex-common cells omitted sex description more often compared to sex-specific cells. None of the cells isolated from embryo and esophagus reported the cell sex in the article. Our results indicate cell sex report in cancer research is limited to a small proportion of cells used in the study. These results call for acknowledging the sex of cells to increase the applicability of biomedical research discoveries.
Collapse
Affiliation(s)
- Mi-Na Park
- Department of Food and Nutrition, Seoul National University, Seoul, Republic of Korea
| | - Sung-Eun Kim
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Sungin Choi
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Yoomee Chang
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Hyeyoon Kim
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Ha-Eun Lee
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Suk Kyeong Lee
- Department of Medical Life Sciences, College of Medicine, the Catholic University of Korea, Seoul, Republic of Korea
- Department of Biomedicine & Health Sciences, College of Medicine, the Catholic University of Korea, Seoul, Republic of Korea
| | - Mi-Kyung Sung
- Department of Food and Nutrition, Sookmyung Women's University, Seoul, Republic of Korea
| | - Hee-Young Paik
- Department of Food and Nutrition, Seoul National University, Seoul, Republic of Korea
- Korea Center for Gendered Innovations in Science and Technology Research, Seoul, Republic of Korea
| |
Collapse
|
14
|
Yang Y, Yang S, Han Y, Zou G, Wang R, Liu L. Quality reporting of randomized controlled trials on SGLT2 inhibitors for heart failure: a comprehensive assessment. Sci Rep 2024; 14:6819. [PMID: 38514865 PMCID: PMC10958037 DOI: 10.1038/s41598-024-57514-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Accepted: 03/19/2024] [Indexed: 03/23/2024] Open
Abstract
Randomised controlled trials (RCTs) provide clinicians with the best evidence of the effectiveness of an intervention, and complete and transparent trial reports help to critically assess and use trial results. The objective of our study was to assess the quality of reporting in RCTs of sodium-glucose co-transporter protein 2 (SGLT2) inhibitors for heart failure (HF) and identify factors associated with improved reporting quality. Two researchers conducted a comprehensive search in four databases (PubMed, Web of Science, EMBASE, and Cochrane). The quality of each report was assessed using a 25-point Overall Quality Score (OQS) based on the guidelines provided in the 2010 Consolidated Standards for Reporting of Trials (CONSORT) statement. We included a total of 58 relevant RCTs. The median OQS in the 2010 CONSORT statement was 15 (range 7.5-24). The missing items were primarily found in the 'Methods' and 'Results' sections of the 2010 CONSORT statement. Multivariate regression modeling revealed that a more recent publication year, high impact factor, and large sample size were significant predictors of OQS improvement. The findings suggest that the overall quality of reported RCTs of SGLT2 inhibitors in HF is unsatisfactory, which reduces their potential usefulness.
Collapse
Affiliation(s)
- YueGuang Yang
- Heilongjiang University of Chinese Medicine, Harbin, Heilongjiang, 150040, People's Republic of China
| | - ShunWen Yang
- Heilongjiang University of Chinese Medicine, Harbin, Heilongjiang, 150040, People's Republic of China
| | - YuBo Han
- The First Department of Cardiovascular, First Affiliated Hospital, Heilongjiang University of Chinese Medicine, 26 Heping Road, Xiangfang, Harbin, Heilongjiang, 150040, People's Republic of China
| | - GuoLiang Zou
- The First Department of Cardiovascular, First Affiliated Hospital, Heilongjiang University of Chinese Medicine, 26 Heping Road, Xiangfang, Harbin, Heilongjiang, 150040, People's Republic of China
| | - RuiNan Wang
- The First Department of Cardiovascular, First Affiliated Hospital, Heilongjiang University of Chinese Medicine, 26 Heping Road, Xiangfang, Harbin, Heilongjiang, 150040, People's Republic of China
| | - Li Liu
- The First Department of Cardiovascular, First Affiliated Hospital, Heilongjiang University of Chinese Medicine, 26 Heping Road, Xiangfang, Harbin, Heilongjiang, 150040, People's Republic of China.
| |
Collapse
|
15
|
Preobrazenski N, McCaig A, Turner A, Kushner M, Pacitti L, Mendolia P, MacDonald B, Storoschuk K, Bouck T, Zaza Y, Lu S, Gurd BJ. Risk of bias in exercise science: A systematic review of 340 studies. iScience 2024; 27:109010. [PMID: 38405604 PMCID: PMC10884506 DOI: 10.1016/j.isci.2024.109010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 12/19/2023] [Accepted: 01/22/2024] [Indexed: 02/27/2024] Open
Abstract
Risk of bias can contribute to irreproducible science and mislead decision making. Analyses of smaller subsections of the exercise science literature suggest many exercise science studies have unclear or high risk of bias. The current review (osf.io/jznv8) assesses whether this unclear or high risk of bias is more widespread in the exercise science literature and whether this bias has decreased since the publication of the 1996 Consolidated Standards of Reporting Trials (CONSORT) guidelines. We report significant reductions in selection, performance, detection, and reporting biases in 2020 compared with 1995 in the 340 of 5,451 studies assessed using the Cochrane Risk of Bias tool. Despite these improvements, most 2020 studies still had unclear or high risks of bias. These results underscore the need for methodological vigilance, adherence to reporting standards, and education on experimental bias. Factors contributing to these improvements, such advancements in education and journal requirements, remain uncertain.
Collapse
Affiliation(s)
| | - Abby McCaig
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Anna Turner
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Maddy Kushner
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Lauren Pacitti
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Peter Mendolia
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Ben MacDonald
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Kristi Storoschuk
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Tori Bouck
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Youssef Zaza
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Stephanie Lu
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| | - Brendon J. Gurd
- School of Kinesiology and Health Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
| |
Collapse
|
16
|
Rodrigues M, Oprea A, Johnson K, Dufort A, Sanger N, Ghiassi P, Sanger S, Panesar B, D'Elia A, Parpia S, Samaan Z, Thabane L. Primary outcome reporting in clinical trials for older adults with depression. BJPsych Open 2024; 10:e60. [PMID: 38450491 PMCID: PMC10951853 DOI: 10.1192/bjo.2023.650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Revised: 11/29/2023] [Accepted: 12/18/2023] [Indexed: 03/08/2024] Open
Abstract
BACKGROUND Findings from randomised controlled trials (RCTs) are synthesised through meta-analyses, which inform evidence-based decision-making. When key details regarding trial outcomes are not fully reported, knowledge synthesis and uptake of findings into clinical practice are impeded. AIMS Our study assessed reporting of primary outcomes in RCTs for older adults with major depressive disorder (MDD). METHOD Trials published between 2011 and 2021, which assessed any intervention for adults aged ≥65 years with a MDD diagnosis, and that specified a single primary outcome were considered for inclusion in our study. Outcome reporting assessment was conducted independently and in duplicate with a 58-item checklist, used in developing the CONSORT-Outcomes statement, and information in each RCT was scored as 'fully reported', 'partially reported' or 'not reported', as applicable. RESULTS Thirty-one of 49 RCTs reported one primary outcome and were included in our study. Most trials (71%) did not fully report over half of the 58 checklist items. Items pertaining to outcome analyses and interpretation were fully reported by 65% or more of trials. Items reported less frequently included: outcome measurement instrument properties (varied from 3 to 30%) and justification of the criteria used to define clinically meaningful change (23%). CONCLUSIONS There is variability in how geriatric depression RCTs report primary outcomes, with omission of details regarding measurement, selection, justification and definition of clinically meaningful change. Outcome reporting deficiencies may hinder replicability and synthesis efforts that inform clinical guidelines and decision-making. The CONSORT-Outcomes guideline should be used when reporting geriatric depression RCTs.
Collapse
Affiliation(s)
- Myanca Rodrigues
- Health Research Methodology Graduate Program, Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada
| | - Anna Oprea
- Life Sciences Undergraduate Program, School of Interdisciplinary Science, McMaster University, Canada
| | - Keily Johnson
- Psychology, Neuroscience and Behaviour Undergraduate Program, Faculty of Science, McMaster University, Canada
| | - Alexander Dufort
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Canada
| | - Nitika Sanger
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Canada
| | - Pegah Ghiassi
- Delivery Management Office, Canadian Partnership Against Cancer, Toronto, Canada
| | | | - Balpreet Panesar
- Neuroscience Graduate Program, McMaster University, Canada; and Department of Psychiatry and Behavioural Neurosciences, St. Joseph's Healthcare Hamilton, Ontario, Canada
| | - Alessia D'Elia
- Neuroscience Graduate Program, McMaster University, Canada; and Department of Psychiatry and Behavioural Neurosciences, St. Joseph's Healthcare Hamilton, Ontario, Canada
| | - Sameer Parpia
- Department of Oncology, McMaster University, Canada; and Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada
| | - Zainab Samaan
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Canada; Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada; and Mood Disorders Program, St. Joseph's Healthcare Hamilton, Ontario, Canada
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada; Population Health Research Institute, Ontario, Canada; and Father Sean O'Sullivan Research Centre, St. Joseph's Healthcare Hamilton, Ontario, Canada
| |
Collapse
|
17
|
Papies EK, Nielsen KS, Soares VA. Health psychology and climate change: time to address humanity's most existential crisis. Health Psychol Rev 2024:1-31. [PMID: 38320578 DOI: 10.1080/17437199.2024.2309242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 01/18/2024] [Indexed: 02/08/2024]
Abstract
Climate change is an ongoing and escalating health emergency. It threatens the health and wellbeing of billions of people, through extreme weather events, displacement, food insecurity, pathogenic diseases, societal destabilisation, and armed conflict. Climate change dwarfs all other challenges studied by health psychologists. The greenhouse gas emissions driving climate change disproportionately originate from the actions of wealthy populations in the Global North and are tied to excessive energy use and overconsumption driven by the pursuit of economic growth. Addressing this crisis requires significant societal transformations and individual behaviour change. Most of these changes will benefit not only the stability of the climate but will yield significant public health co-benefits. Because of their unique expertise and skills, health psychologists are urgently needed in crafting climate change mitigation responses. We propose specific ways in which health psychologists at all career stages can contribute, within the spheres of research, teaching, and policy making, and within organisations and as private citizens. As health psychologists, we cannot sit back and leave climate change to climate scientists. Climate change is a health emergency that results from human behaviour; hence it is in our power and responsibility to address it.
Collapse
Affiliation(s)
- Esther K Papies
- MRC/CSO Social and Public Health Sciences Unit, School of Health and Wellbeing, University of Glasgow, United Kingdom
| | | | | |
Collapse
|
18
|
Zrubka Z, Kertész G, Gulácsi L, Czere J, Hölgyesi Á, Nezhad HM, Mosavi A, Kovács L, Butte AJ, Péntek M. The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review. J Med Internet Res 2024; 26:e47430. [PMID: 38241075 PMCID: PMC10837761 DOI: 10.2196/47430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 04/29/2023] [Accepted: 11/17/2023] [Indexed: 01/23/2024] Open
Abstract
BACKGROUND Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine. OBJECTIVE We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies. METHODS We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility. RESULTS After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant. CONCLUSIONS The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.
Collapse
Affiliation(s)
- Zsombor Zrubka
- HECON Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
| | - Gábor Kertész
- John von Neumann Faculty of Informatics, Óbuda University, Budapest, Hungary
| | - László Gulácsi
- HECON Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
| | - János Czere
- Doctoral School of Innovation Management, Óbuda University, Budapest, Hungary
| | - Áron Hölgyesi
- HECON Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
- Doctoral School of Molecular Medicine, Semmelweis University, Budapest, Hungary
| | - Hossein Motahari Nezhad
- HECON Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
- Doctoral School of Business and Management, Corvinus University of Budapest, Budapest, Hungary
| | - Amir Mosavi
- John von Neumann Faculty of Informatics, Óbuda University, Budapest, Hungary
| | - Levente Kovács
- Physiological Controls Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
| | - Atul J Butte
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, United States
| | - Márta Péntek
- HECON Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
| |
Collapse
|
19
|
Zhang J, Deng J, Wang N, Wang P, Li J, Wang Y, Cui W, Liang F, Chen P, Wang J, Han F, Chan CP, Lyu A, Bian Z, Zhang X. Quality of reporting of integrative Chinese and Western medicine intervention in randomized controlled trials of ulcerative colitis: a review. Syst Rev 2023; 12:228. [PMID: 38062492 PMCID: PMC10704853 DOI: 10.1186/s13643-023-02402-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 11/29/2023] [Indexed: 12/18/2023] Open
Abstract
BACKGROUND Integrative Chinese and Western medicine (ICWM) is commonly used for the treatment of ulcerative colitis (UC) in clinical practice. However, it is unclear whether the details of ICWM interventions, such as selection rationale, implementation design, and potential interactions, were adequately reported. Therefore, this study aimed to assess the quality of reporting in the ICWM interventional randomized controlled trials (RCTs) of UC and to identify the common problems if any. METHODS Through a search of 10 international electronic databases, we identified RCTs of UC with ICWM interventions published in English or Chinese from the inception date of each database up to 16 June 2023. Literature screening was strictly conducted based on the inclusion and exclusion criteria of the Population, Concept, and Context (PCC) framework. The general characteristics of the included studies were described. The quality of reporting was assessed according to three checklists, including the CONSORT (Consolidated Standards of Reporting Trials) with 36 items (except for one item 1b about abstract), the CONSORT for Abstracts (17 items), and a self-designed ICWM-related checklist (27 items covering design rationale, intervention details, outcome assessments, and analysis). The reporting scores of RCTs published before and after 2010 were compared. RESULTS A total of 1458 eligible RCTs were included. For the reporting compliance, the median score (interquartile ranges) of the CONSORT (72 score in total), the CONSORT for Abstract (34 score), and ICWM-related (54 score) items was 21 (18-25), 13 (12-15), and 18 (15-21), respectively. Although the time period comparisons showed that reporting quality of included publications improved significantly after the CONSORT 2010 issued (P < 0.01), more than 50% of items were evaluated as poor quality (reporting rate < 65%) among each checklist, especially in the CONSORT for Abstract and ICWM-specific items. CONCLUSION Although CONSORT appears to have enhanced the reporting of RCTs in UC, the quality of ICWM specifics is variable and in need of improvement. Reporting guidelines of the ICWM recommendations should be developed to improve their quality.
Collapse
Affiliation(s)
- Jialing Zhang
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
- Centre for Chinese Herbal Medicine Drug Development, Hong Kong Baptist University, HKSAR, China
| | - Jiashuai Deng
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Nana Wang
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
| | - Ping Wang
- China Academy of Chinese Medical Sciences, Xiyuan Hospital, Beijing, China
| | - Ji Li
- Department of Pediatrics, China Academy of Chinese Medical Sciences, Guang'anmen Hospital, Beijing, China
| | - Yunhai Wang
- College of Traditional Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Wanting Cui
- College of Traditional Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Feng Liang
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
| | - Peijin Chen
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
| | - Juan Wang
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
| | - Fei Han
- Department of Pediatrics, China Academy of Chinese Medical Sciences, Guang'anmen Hospital, Beijing, China
| | - Chun Pong Chan
- Department of Computer Science, Faculty of Science, Hong Kong Baptist University, HKSAR, China
| | - Aiping Lyu
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China
| | - Zhaoxiang Bian
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China.
- Centre for Chinese Herbal Medicine Drug Development, Hong Kong Baptist University, HKSAR, China.
| | - Xuan Zhang
- Hong Kong Chinese Medicine Clinical Study Centre, Chinese EQUATOR Centre, School of Chinese Medicine, Chinese Clinical Trial Registry (Hong Kong), Hong Kong Baptist University, HKSAR, China.
- Centre for Chinese Herbal Medicine Drug Development, Hong Kong Baptist University, HKSAR, China.
| |
Collapse
|
20
|
Demetriades AK, Tiefenbach J, Park JJ, Anwar MM, Raza SM. What is the quality of reporting in randomized controlled trials in spinal conditions. JOURNAL OF CRANIOVERTEBRAL JUNCTION AND SPINE 2023; 14:404-411. [PMID: 38268696 PMCID: PMC10805167 DOI: 10.4103/jcvjs.jcvjs_121_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 01/26/2024] Open
Abstract
Purpose Substandard quality across published randomized controlled trials (RCTs) is a major concern. Imperfect reporting has the potential to distort the evidence landscape and waste valuable health-care resources. In this study, we aim to assess the current quality of reporting in the field of spine using a modified version of the Consolidated Standards of Reporting Trials (CONSORT) checklist. Materials and Methods A list of published RCTs in the field of spine disease from January 1, 2013, to December 31, 2020, was built. Two reviewers scored the published RCTs against a modified CONSORT checklist. The mean adjusted CONSORT scores for each study, reporting category, and checklist item were calculated. Results The mean and median scores across all of the RCTs were 0.72 and 0.74 out of 1.00, respectively. The spectrum of scores was wide, ranging from 0.45 to 0.94. The reporting categories with the lowest score included randomization, blinding, and abstract. The items which were most under-reported included allocation sequence generation, type of randomization used, full trial protocol details, and abstract methodology. The inter-rater reliability between our reviewers was substantial (κ = 0.7, κ = 0.71). Conclusion Our findings correlate with only a moderate level of compliance to the CONSORT criteria on the quality of reporting for RCTs in spinal conditions. This is in line with previous reports on compliance, both within and outside the field of spinal conditions. Further continued and sustained efforts are still required to enhance the quality and consistency of RCT reporting, ultimately reducing health-care resource wastage and improving patient safety.
Collapse
Affiliation(s)
- Andreas K. Demetriades
- Department of Neurosurgery, University of Leiden, Leiden, The Netherlands
- Department of Clinical Neurosciences, Royal Infirmary of Edinburgh, Edinburgh, UK
- Edinburgh Spinal Surgery Outcomes Studies Group, Edinburgh, UK
| | - Jakov Tiefenbach
- Edinburgh Spinal Surgery Outcomes Studies Group, Edinburgh, UK
- Neurological Institute, Cleveland Clinic, Ohio, USA
| | - Jay J. Park
- Edinburgh Spinal Surgery Outcomes Studies Group, Edinburgh, UK
- Edinburgh Medical School, University of Edinburgh, Edinburgh, UK
| | | | | |
Collapse
|
21
|
Pfledderer CD, von Klinggraeff L, Burkart S, da Silva Bandeira A, Armstrong B, Weaver RG, Adams EL, Beets MW. Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review. Pilot Feasibility Stud 2023; 9:161. [PMID: 37705118 PMCID: PMC10498529 DOI: 10.1186/s40814-023-01389-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 08/29/2023] [Indexed: 09/15/2023] Open
Abstract
BACKGROUND Guidelines, checklists, frameworks, and recommendations (GCFRs) related to preliminary studies serve as essential resources to assist behavioral intervention researchers in reporting findings from preliminary studies, but their impact on preliminary study reporting comprehensiveness is unknown. The purpose of this study was to conduct a scoping bibliometric review of recently published preliminary behavioral-focused intervention studies to (1) examine the prevalence of GCFR usage and (2) determine the associations between GCFR usage and reporting feasibility-related characteristics. METHODS A systematic search was conducted for preliminary studies of behavioral-focused interventions published between 2018 and 2020. Studies were limited to the top 25 journals publishing behavioral-focused interventions, text mined to identify usage of GCFRs, and categorized as either not citing GCFRs or citing ≥ 2 GCFRs (Citers). A random sample of non-Citers was text mined to identify studies which cited other preliminary studies that cited GCFRs (Indirect Citers) and those that did not (Never Citers). The presence/absence of feasibility-related characteristics was compared between Citers, Indirect Citers, and Never Citers via univariate logistic regression. RESULTS Studies (n = 4143) were identified, and 1316 were text mined to identify GCFR usage (n = 167 Citers). A random sample of 200 studies not citing a GCFR were selected and categorized into Indirect Citers (n = 71) and Never Citers (n = 129). Compared to Never Citers, Citers had higher odds of reporting retention, acceptability, adverse events, compliance, cost, data collection feasibility, and treatment fidelity (ORrange = 2.62-14.15, p < 0.005). Citers also had higher odds of mentioning feasibility in purpose statements, providing progression criteria, framing feasibility as the primary outcome, and mentioning feasibility in conclusions (ORrange = 6.31-17.04, p < 0.005) and lower odds of mentioning efficacy in purpose statements, testing for efficacy, mentioning efficacy in conclusions, and suggesting future testing (ORrange = 0.13-0.54, p < 0.05). Indirect Citers had higher odds of reporting acceptability and treatment fidelity (ORrange = 2.12-2.39, p < 0.05) but lower odds of testing for efficacy (OR = 0.36, p < 0.05) compared to Never Citers. CONCLUSION The citation of GCFRs is associated with greater reporting of feasibility-related characteristics in preliminary studies of behavioral-focused interventions. Researchers are encouraged to use and cite literature that provides guidance on design, implementation, analysis, and reporting to improve the comprehensiveness of reporting for preliminary studies.
Collapse
Affiliation(s)
- Christopher D Pfledderer
- Department of Health Promotion and Behavioral Sciences, The University of Texas Health Science Center at Houston, School of Public Health Austin Campus, Austin, TX, 78701, USA.
| | - Lauren von Klinggraeff
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| | - Sarah Burkart
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| | | | - Bridget Armstrong
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| | - R Glenn Weaver
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| | - Elizabeth L Adams
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| | - Michael W Beets
- Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA
| |
Collapse
|
22
|
Pronk AJM, Roelofs A, Flum DR, Bonjer HJ, Abu Hilal M, Dijkgraaf MGW, Besselink MG, Ahmed Ali U. Two decades of surgical randomized controlled trials: worldwide trends in volume and methodological quality. Br J Surg 2023; 110:1300-1308. [PMID: 37379487 PMCID: PMC10480038 DOI: 10.1093/bjs/znad160] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 03/07/2023] [Accepted: 05/04/2023] [Indexed: 06/30/2023]
Abstract
BACKGROUND RCTs are essential in guiding clinical decision-making but are difficult to perform, especially in surgery. This review assessed the trend in volume and methodological quality of published surgical RCTs over two decades. METHODS PubMed was searched systematically for surgical RCTs published in 1999, 2009, and 2019. The primary outcomes were volume of trials and RCTs with a low risk of bias. Secondary outcomes were clinical, geographical, and funding characteristics. RESULTS Some 1188 surgical RCTs were identified, of which 300 were published in 1999, 450 in 2009, and 438 in 2019. The most common subspecialty in 2019 was gastrointestinal surgery (50.7 per cent). The volume of surgical RCTs increased mostly in Asia (61, 159, and 199 trials), especially in China (7, 40, and 81). In 2019, countries with the highest relative volume of published surgical RCTs were Finland and the Netherlands. Between 2009 and 2019, the proportion of RCTs with a low risk of bias increased from 14.7 to 22.1 per cent (P = 0.004). In 2019, the proportion of trials with a low risk of bias was highest in Europe (30.5 per cent), with the UK and the Netherlands as leaders in this respect. CONCLUSION The volume of published surgical RCTs worldwide remained stable in the past decade but their methodological quality improved. Considerable geographical shifts were observed, with Asia and especially China leading in terms of volume. Individual European countries are leading in their relative volume and methodological quality of surgical RCTs.
Collapse
Affiliation(s)
- Aagje J M Pronk
- Department of Surgery, Amsterdam UMC, location University of Amsterdam, Amsterdam, the Netherlands
- Cancer Centre Amsterdam, Amsterdam, the Netherlands
| | - Anne Roelofs
- Department of Surgery, Amsterdam UMC, location University of Amsterdam, Amsterdam, the Netherlands
- Cancer Centre Amsterdam, Amsterdam, the Netherlands
| | - David R Flum
- Department of Surgery, University of Washington, Seattle, Washington, USA
| | - H Jaap Bonjer
- Cancer Centre Amsterdam, Amsterdam, the Netherlands
- Department of Surgery, Amsterdam UMC, location Vrije Universiteit, Amsterdam, the Netherlands
| | - Mohammed Abu Hilal
- Department of Surgery, Fondazione Poliambulanza Hospital, Brescia, Italy
| | - Marcel G W Dijkgraaf
- Epidemiology and Data Science, Amsterdam UMC, location University of Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Amsterdam, the Netherlands
| | - Marc G Besselink
- Department of Surgery, Amsterdam UMC, location University of Amsterdam, Amsterdam, the Netherlands
- Cancer Centre Amsterdam, Amsterdam, the Netherlands
| | - Usama Ahmed Ali
- Department of Surgery, Amsterdam UMC, location University of Amsterdam, Amsterdam, the Netherlands
- Cancer Centre Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
23
|
Lalanza JF, Lorente S, Bullich R, García C, Losilla JM, Capdevila L. Methods for Heart Rate Variability Biofeedback (HRVB): A Systematic Review and Guidelines. Appl Psychophysiol Biofeedback 2023; 48:275-297. [PMID: 36917418 PMCID: PMC10412682 DOI: 10.1007/s10484-023-09582-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/15/2023] [Indexed: 03/16/2023]
Abstract
Heart Rate Variability Biofeedback (HRVB) has been widely used to improve cardiovascular health and well-being. HRVB is based on breathing at an individual's resonance frequency, which stimulates respiratory sinus arrhythmia (RSA) and the baroreflex. There is, however, no methodological consensus on how to apply HRVB, while details about the protocol used are often not well reported. Thus, the objectives of this systematic review are to describe the different HRVB protocols and detect methodological concerns. PsycINFO, CINALH, Medline and Web of Science were searched between 2000 and April 2021. Data extraction and quality assessment were based on PRISMA guidelines. A total of 143 studies were finally included from any scientific field and any type of sample. Three protocols for HRVB were found: (i) "Optimal RF" (n = 37), each participant breathes at their previously detected RF; (ii) "Individual RF" (n = 48), each participant follows a biofeedback device that shows the optimal breathing rate based on cardiovascular data in real time, and (iii) "Preset-pace RF" (n = 51), all participants breathe at the same rate rate, usually 6 breaths/minute. In addition, we found several methodological differences for applying HRVB in terms of number of weeks, duration of breathing or combination of laboratory and home sessions. Remarkably, almost 2/3 of the studies did not report enough information to replicate the HRVB protocol in terms of breathing duration, inhalation/exhalation ratio, breathing control or body position. Methodological guidelines and a checklist are proposed to enhance the methodological quality of future HRVB studies and increase the information reported.
Collapse
Affiliation(s)
- Jaume F Lalanza
- Department of Basic Psychology, Universitat Autònoma de Barcelona, Bellaterra, Spain
- Department of Psychology, UiT The Arctic University of Norway, Tromsø, Norway
| | - Sonia Lorente
- Department of Psychobiology and Methodology of Health Science, Universitat Autònoma de Barcelona, Bellaterra, Spain
- Pediatric Area, Hospital de Terrassa, Consorci Sanitari de Terrassa, Terrassa, Spain
| | - Raimon Bullich
- Department of Basic Psychology, Universitat Autònoma de Barcelona, Bellaterra, Spain
| | - Carlos García
- Department of Basic Psychology, Universitat Autònoma de Barcelona, Bellaterra, Spain
| | - Josep-Maria Losilla
- Department of Psychobiology and Methodology of Health Science, Universitat Autònoma de Barcelona, Bellaterra, Spain
- Sport Research Institute UAB, Universitat Autònoma de Barcelona, Bellaterra, Spain
| | - Lluis Capdevila
- Department of Basic Psychology, Universitat Autònoma de Barcelona, Bellaterra, Spain.
- Sport Research Institute UAB, Universitat Autònoma de Barcelona, Bellaterra, Spain.
- Departament of Basic Psychology, Universitat Autònoma de Barcelona, 08193, Bellaterra, Barcelona, Spain.
| |
Collapse
|
24
|
Klement W, El Emam K. Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Modeling Studies: Development and Validation. J Med Internet Res 2023; 25:e48763. [PMID: 37651179 PMCID: PMC10502599 DOI: 10.2196/48763] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 07/11/2023] [Accepted: 07/31/2023] [Indexed: 09/01/2023] Open
Abstract
BACKGROUND The reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements. OBJECTIVE We aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies. METHODS We conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average κ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items. RESULTS In total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting. CONCLUSIONS Our findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies.
Collapse
Affiliation(s)
- William Klement
- University of Ottawa, Ottawa, ON, Canada
- CHEO Research Institute, Ottawa, ON, Canada
| | - Khaled El Emam
- University of Ottawa, Ottawa, ON, Canada
- CHEO Research Institute, Ottawa, ON, Canada
| |
Collapse
|
25
|
Harrer M, Cuijpers P, Schuurmans LKJ, Kaiser T, Buntrock C, van Straten A, Ebert D. Evaluation of randomized controlled trials: a primer and tutorial for mental health researchers. Trials 2023; 24:562. [PMID: 37649083 PMCID: PMC10469910 DOI: 10.1186/s13063-023-07596-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 08/18/2023] [Indexed: 09/01/2023] Open
Abstract
BACKGROUND Considered one of the highest levels of evidence, results of randomized controlled trials (RCTs) remain an essential building block in mental health research. They are frequently used to confirm that an intervention "works" and to guide treatment decisions. Given their importance in the field, it is concerning that the quality of many RCT evaluations in mental health research remains poor. Common errors range from inadequate missing data handling and inappropriate analyses (e.g., baseline randomization tests or analyses of within-group changes) to unduly interpretations of trial results and insufficient reporting. These deficiencies pose a threat to the robustness of mental health research and its impact on patient care. Many of these issues may be avoided in the future if mental health researchers are provided with a better understanding of what constitutes a high-quality RCT evaluation. METHODS In this primer article, we give an introduction to core concepts and caveats of clinical trial evaluations in mental health research. We also show how to implement current best practices using open-source statistical software. RESULTS Drawing on Rubin's potential outcome framework, we describe that RCTs put us in a privileged position to study causality by ensuring that the potential outcomes of the randomized groups become exchangeable. We discuss how missing data can threaten the validity of our results if dropouts systematically differ from non-dropouts, introduce trial estimands as a way to co-align analyses with the goals of the evaluation, and explain how to set up an appropriate analysis model to test the treatment effect at one or several assessment points. A novice-friendly tutorial is provided alongside this primer. It lays out concepts in greater detail and showcases how to implement techniques using the statistical software R, based on a real-world RCT dataset. DISCUSSION Many problems of RCTs already arise at the design stage, and we examine some avoidable and unavoidable "weak spots" of this design in mental health research. For instance, we discuss how lack of prospective registration can give way to issues like outcome switching and selective reporting, how allegiance biases can inflate effect estimates, review recommendations and challenges in blinding patients in mental health RCTs, and describe problems arising from underpowered trials. Lastly, we discuss why not all randomized trials necessarily have a limited external validity and examine how RCTs relate to ongoing efforts to personalize mental health care.
Collapse
Affiliation(s)
- Mathias Harrer
- Psychology and Digital Mental Health Care, Technical University Munich, Georg-Brauchle-Ring 60-62, Munich, 80992, Germany.
- Clinical Psychology and Psychotherapy, Institute for Psychology, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany.
| | - Pim Cuijpers
- Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- WHO Collaborating Centre for Research and Dissemination of Psychological Interventions, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Lea K J Schuurmans
- Psychology and Digital Mental Health Care, Technical University Munich, Georg-Brauchle-Ring 60-62, Munich, 80992, Germany
| | - Tim Kaiser
- Methods and Evaluation/Quality Assurance, Freie Universität Berlin, Berlin, Germany
| | - Claudia Buntrock
- Institute of Social Medicine and Health Systems Research (ISMHSR), Medical Faculty, Otto Von Guericke University Magdeburg, Magdeburg, Germany
| | - Annemieke van Straten
- Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - David Ebert
- Psychology and Digital Mental Health Care, Technical University Munich, Georg-Brauchle-Ring 60-62, Munich, 80992, Germany
| |
Collapse
|
26
|
van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane PJ. Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework. J Am Med Inform Assoc 2023; 30:1503-1515. [PMID: 37208863 PMCID: PMC10436156 DOI: 10.1093/jamia/ocad088] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 04/17/2023] [Accepted: 05/09/2023] [Indexed: 05/21/2023] Open
Abstract
OBJECTIVE To derive a comprehensive implementation framework for clinical AI models within hospitals informed by existing AI frameworks and integrated with reporting standards for clinical AI research. MATERIALS AND METHODS (1) Derive a provisional implementation framework based on the taxonomy of Stead et al and integrated with current reporting standards for AI research: TRIPOD, DECIDE-AI, CONSORT-AI. (2) Undertake a scoping review of published clinical AI implementation frameworks and identify key themes and stages. (3) Perform a gap analysis and refine the framework by incorporating missing items. RESULTS The provisional AI implementation framework, called SALIENT, was mapped to 5 stages common to both the taxonomy and the reporting standards. A scoping review retrieved 20 studies and 247 themes, stages, and subelements were identified. A gap analysis identified 5 new cross-stage themes and 16 new tasks. The final framework comprised 5 stages, 7 elements, and 4 components, including the AI system, data pipeline, human-computer interface, and clinical workflow. DISCUSSION This pragmatic framework resolves gaps in existing stage- and theme-based clinical AI implementation guidance by comprehensively addressing the what (components), when (stages), and how (tasks) of AI implementation, as well as the who (organization) and why (policy domains). By integrating research reporting standards into SALIENT, the framework is grounded in rigorous evaluation methodologies. The framework requires validation as being applicable to real-world studies of deployed AI models. CONCLUSIONS A novel end-to-end framework has been developed for implementing AI within hospital clinical practice that builds on previous AI implementation frameworks and research reporting standards.
Collapse
Affiliation(s)
- Anton H van der Vegt
- Centre for Health Services Research, The University of Queensland, Brisbane, Australia
| | - Ian A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Australia
| | - Krishna Dermawan
- Centre for Information Resilience, The University of Queensland, St Lucia, Australia
| | - Rudolf J Schnetler
- School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Australia
| | - Vikrant R Kalke
- Patient Safety and Quality, Clinical Excellence Queensland, Queensland Health, Brisbane, Australia
| | - Paul J Lane
- Safety Quality & Innovation, The Prince Charles Hospital, Queensland Health, Brisbane, Australia
| |
Collapse
|
27
|
Rahman E, Philip-Dormston WG, Webb WR, Rao P, Carruthers JD, Carruthers A, Swift A, Goodman GJ, Mosahebi A, Nahai F. Developing Consensus-Based Guidelines for Case Reporting in Aesthetic Medicine: Enhancing Transparency and Standardization. Aesthet Surg J Open Forum 2023; 5:ojad076. [PMID: 37694226 PMCID: PMC10483583 DOI: 10.1093/asjof/ojad076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/12/2023] Open
Abstract
Clinical case reporting plays a vital role in sharing detailed patient narratives, providing insights into rare conditions, innovative treatments, and unexpected outcomes. However, existing reporting guidelines in aesthetic medicine fail to capture the specific nuances of procedures and outcomes in this field. The authors' objectives were to develop comprehensive guidelines for Case REporting in Aesthetic Medicine (CREAM). The study employed a 3-phase consensus process, including a literature review, expert interviews, and a consensus meeting. A diverse group of 10 expert participants (plastic surgeons, dermatologists, noncore specialists, evidence-based medicine expert, and research scientist) in Phase I and 30 experienced aesthetic practitioners in Phase II contributed to the research. Statistical analysis was conducted to assess agreement levels among participants and explore associations and variations within the data. The participants represented various specialties, genders, LGBTQ+ identities, and ethnic backgrounds. The research resulted in the development of the CREAM guidelines, consisting of a 16-item checklist. The guidelines covered essential aspects of case reporting, such as patient and practice information, procedure details, clinical assessment and outcomes, adverse events, and ethical considerations. Statistical analysis indicated a high level of consensus among participants, as well as significant associations between checklist items. CREAM guidelines represent a step toward enhancing transparency and standardization in case reporting in aesthetic medicine. Adhering to these guidelines will allow authors to contribute to a robust evidence base, prioritize patient safety, and drive advancements aesthetic medicine.
Collapse
Affiliation(s)
- Eqram Rahman
- Corresponding Author: Dr Eqram Rahman, Pond Street, Hampstead NW3 2QG, UK. E-mail: ; Instagram: @Beyond_border_Aesthetics
| | | | | | | | | | | | | | | | | | | |
Collapse
|
28
|
van Ravenzwaaij D, Bakker M, Heesen R, Romero F, van Dongen N, Crüwell S, Field SM, Held L, Munafò MR, Pittelkow MM, Tiokhin L, Traag VA, van den Akker OR, van ‘t Veer AE, Wagenmakers EJ. Perspectives on scientific error. ROYAL SOCIETY OPEN SCIENCE 2023; 10:230448. [PMID: 37476516 PMCID: PMC10354464 DOI: 10.1098/rsos.230448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 06/27/2023] [Indexed: 07/22/2023]
Abstract
Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.
Collapse
Affiliation(s)
- D. van Ravenzwaaij
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, Heymans Building, room 239, 9712 TS Groningen, The Netherlands
| | - M. Bakker
- Tilburg University, 5037 AB Tilburg, The Netherlands
| | - R. Heesen
- University of Western Australia, Perth, Western Australia 6009, Australia
- London School of Economics and Political Science, London WC2A 2AE, UK
| | - F. Romero
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, Heymans Building, room 239, 9712 TS Groningen, The Netherlands
| | - N. van Dongen
- University of Amsterdam, 1012 WP Amsterdam, The Netherlands
| | - S. Crüwell
- Department of History and Philosophy of Science, University of Cambridge, Cambridge CB2 1TN, UK
| | - S. M. Field
- Centre for Science and Technology Studies (CWTS), Leiden University, 2311 EZ Leiden, The Netherlands
| | - L. Held
- University of Zurich, 8006 Zürich, Switzerland
| | - M. R. Munafò
- School of Psychological Science, University of Bristol, Bristol BS8 1QU, UK
| | - M. M. Pittelkow
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, Heymans Building, room 239, 9712 TS Groningen, The Netherlands
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité—Universitätsmedizin, 10178 Berlin, Germany
| | - L. Tiokhin
- IG&H Consulting, 3528 AC Utrecht, The Netherlands
| | - V. A. Traag
- Centre for Science and Technology Studies (CWTS), Leiden University, 2311 EZ Leiden, The Netherlands
| | - O. R. van den Akker
- Tilburg University, 5037 AB Tilburg, The Netherlands
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité—Universitätsmedizin, 10178 Berlin, Germany
| | - A. E. van ‘t Veer
- Methodology and Statistics Unit, Institute of Psychology, Leiden University, 2333 AK Leiden, The Netherlands
| | | |
Collapse
|
29
|
Pamporis K, Bougioukas KI, Karakasis P, Papageorgiou D, Zarifis I, Haidich AB. Overviews of reviews in the cardiovascular field underreported critical methodological and transparency characteristics: a methodological study based on the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement. J Clin Epidemiol 2023; 159:139-150. [PMID: 37245702 DOI: 10.1016/j.jclinepi.2023.05.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Revised: 05/17/2023] [Accepted: 05/22/2023] [Indexed: 05/30/2023]
Abstract
OBJECTIVES This study aimed to evaluate the epidemiology, reporting characteristics, and adherence to the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement of overviews of reviews (overviews) of interventions in the cardiovascular field. STUDY DESIGN AND SETTING MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched from January 1, 2000, to October 15, 2020. An updated search was performed in MEDLINE, Epistemonikos, and Google Scholar up to August 25, 2022. Overviews of interventions published in English and primarily considering populations, interventions, and outcomes pertinent to the cardiovascular field were eligible. Study selection, data extraction, and PRIOR adherence assessment were performed by two authors independently. RESULTS We analyzed 96 overviews. Almost half (43/96 [45%]) were published between 2020 and 2022 and contained a median of 15 systematic reviews (SRs) (interquartile range, 9-28). The commonest title terminology was "overview of (systematic) reviews" (38/96 [40%]). Methods for handling SR overlap were reported in 24/96 (25%), methods for assessing primary study overlap in 18/96 (19%), handling of discrepant data in 11/96 (11%), and methods for methodological quality or risk of bias assessment of the primary studies within SRs in 23/96 (24%). Authors included data sharing statements in 28/96 (29%), complete funding disclosure in 43/96 (45%), protocol registration in 43/96 (45%), and conflict of interest statement in 82/96 (85%) overviews. CONCLUSION Insufficient reporting was identified in methodological characteristics unique in overviews' conduct and most transparency markers. Adoption of PRIOR from the research community could ameliorate overviews' reporting.
Collapse
Affiliation(s)
- Konstantinos Pamporis
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece
| | - Konstantinos I Bougioukas
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece
| | - Paschalis Karakasis
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece
| | - Dimitrios Papageorgiou
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece
| | - Ippokratis Zarifis
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece
| | - Anna-Bettina Haidich
- Department of Hygiene, Social-Preventive Medicine & Medical Statistics, Medical School, Aristotle University of Thessaloniki, University Campus, Thessaloniki 54124, Greece.
| |
Collapse
|
30
|
Zhao R, Zhang W, Zhang Z, He C, Xu R, Tang X, Wang B. Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review. BMC Med Res Methodol 2023; 23:152. [PMID: 37386371 PMCID: PMC10308622 DOI: 10.1186/s12874-023-01960-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 05/31/2023] [Indexed: 07/01/2023] Open
Abstract
OBJECTIVE Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors. METHODS We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time. RESULTS 187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles. CONCLUSION The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.
Collapse
Affiliation(s)
- Ran Zhao
- Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Wen Zhang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - ZeDan Zhang
- Traditional Chinese Medicine Data Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Chang He
- Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Rong Xu
- Guang'anmeng Hospital, China Academy of Chinese Medical Sciences, Beijing, China
| | - XuDong Tang
- China Academy of Chinese Medical Sciences, Beijing, China.
| | - Bin Wang
- Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing, China.
- Traditional Chinese Medicine Data Center, China Academy of Chinese Medical Sciences, Beijing, China.
| |
Collapse
|
31
|
Daykin GM, Aizen MA, Barrett LG, Bartlett LJ, Batáry P, Garibaldi LA, Güncan A, Gutam S, Maas B, Mitnala J, Montaño-Centellas F, Muoni T, Öckinger E, Okechalu O, Ostler R, Potts SG, Rose DC, Topp CFE, Usieta HO, Utoblo OG, Watson C, Zou Y, Sutherland WJ, Hood ASC. AgroEcoList 1.0: A checklist to improve reporting standards in ecological research in agriculture. PLoS One 2023; 18:e0285478. [PMID: 37310957 DOI: 10.1371/journal.pone.0285478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Accepted: 04/24/2023] [Indexed: 06/15/2023] Open
Abstract
Many publications lack sufficient background information (e.g. location) to be interpreted, replicated, or reused for synthesis. This impedes scientific progress and the application of science to practice. Reporting guidelines (e.g. checklists) improve reporting standards. They have been widely taken up in the medical sciences, but not in ecological and agricultural research. Here, we use a community-centred approach to develop a reporting checklist (AgroEcoList 1.0) through surveys and workshops with 23 experts and the wider agroecological community. To put AgroEcoList in context, we also assessed the agroecological community's perception of reporting standards in agroecology. A total of 345 researchers, reviewers, and editors, responded to our survey. Although only 32% of respondents had prior knowledge of reporting guidelines, 76% of those that had said guidelines improved reporting standards. Overall, respondents agreed on the need of AgroEcolist 1.0; only 24% of respondents had used reporting guidelines before, but 78% indicated they would use AgroEcoList 1.0. We updated AgroecoList 1.0 based on respondents' feedback and user-testing. AgroecoList 1.0 consists of 42 variables in seven groups: experimental/sampling set-up, study site, soil, livestock management, crop and grassland management, outputs, and finances. It is presented here, and is also available on github (https://github.com/AgroecoList/Agroecolist). AgroEcoList 1.0 can serve as a guide for authors, reviewers, and editors to improve reporting standards in agricultural ecology. Our community-centred approach is a replicable method that could be adapted to develop reporting checklists in other fields. Reporting guidelines such as AgroEcoList can improve reporting standards and therefore the application of research to practice, and we recommend that they are adopted more widely in agriculture and ecology.
Collapse
Affiliation(s)
- Georgia M Daykin
- Department of Zoology, University of Cambridge, Cambridge, United Kingdom
| | - Marcelo A Aizen
- Instituto de Investigaciones en Biodiversidad y Medio Ambiente (INIBIOMA), Universidad Nacional del Comahue - Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), San Carlos de Bariloche, Río Negro, Argentina
| | | | - Lewis J Bartlett
- Center for the Ecology of Infectious Diseases, Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
| | - Péter Batáry
- "Lendület" Landscape and Conservation Ecology, Institute of Ecology and Botany, Centre for Ecological Research, Vácrátót, Alkomány, Hungary
| | - Lucas A Garibaldi
- Instituto de Investigaciones en Recursos Naturales, Agroecología y Desarrollo Rural, Universidad Nacional de Río Negro, Viedma, Río Negro, Argentina
- Consejo Nacional de Investigaciones Científicas y Técnicas, Instituto de Investigaciones en Recursos Naturales, Agroecología y Desarrollo Rural, Bariloche, Río Negro, Argentina
| | - Ali Güncan
- Department of Plant Protection, Faculty of Agriculture, University of Ordu, Ordu, Turkey
| | - Sridhar Gutam
- ICAR-AICRP on Fruits, ICAR-Indian Institute of Horticultural Research, Bengaluru, Karnataka, India
| | - Bea Maas
- Department of Botany and Biodiversity Research, University of Vienna, Vienna, Austria
- Agroecology, University of Goettingen, Göettingen, Germany
| | - Jayalakshmi Mitnala
- Regional Agricultural Research Station, Acharya N. G. Ranga Agricultural University, Hyderabad, Andhra Pradesh, India
| | - Flavia Montaño-Centellas
- Instituto de Ecología, Universidad Mayor de San Andrés, La Paz, Bolivia
- Department of Biological Sciences, Louisiana State University, Baton Rouge, Louisiana, United States of America
| | - Tarirai Muoni
- CIMMYT Southern Africa Regional Office, Harare, Zimbabwe
- Department of Crop Production Ecology, Swedish University of Agricultural Sciences, Uppsala, Sweden
| | - Erik Öckinger
- Department of Ecology, Swedish University of Agricultural Sciences, Uppsala, Sweden
| | - Ode Okechalu
- Department of Plant Science and Biotechnology, University of Jos, Plateau, Nigeria
| | - Richard Ostler
- Computational and Analytical Sciences, Rothamsted Research, Harpenden, United Kingdom
| | - Simon G Potts
- Centre for Agri-environmental Research, School of Agriculture, Policy and Development, University of Reading, Reading, United Kingdom
| | - David C Rose
- Centre for Agri-environmental Research, School of Agriculture, Policy and Development, University of Reading, Reading, United Kingdom
- School of Water, Energy, and Environment, Cranfield University, Cranfield, United Kingdom
| | - Cairistiona F E Topp
- Agriculture, Horticulture and Engineering Sciences, Scotland's Rural College, Edinburgh, United Kingdom
| | - Hope O Usieta
- Leventis Foundation Nigeria, F. C. T. Abuja, Nigeria
| | - Obaiya G Utoblo
- Department of Plant Science and Biotechnology, University of Jos, Plateau, Nigeria
| | - Christine Watson
- Department of Crop Production Ecology, Swedish University of Agricultural Sciences, Uppsala, Sweden
- Rural Land Use, Scotland's Rural College, Craibstone Estate, Aberdeen, United Kingdom
| | - Yi Zou
- Department of Health and Environmental Sciences, Xi'an Jiaotong-Liverpool University, Suzhou, P. R. China
| | | | - Amelia S C Hood
- Department of Zoology, University of Cambridge, Cambridge, United Kingdom
- Centre for Agri-environmental Research, School of Agriculture, Policy and Development, University of Reading, Reading, United Kingdom
| |
Collapse
|
32
|
Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier-Hein L, Mercaldo N, Müller H, Orlhac F, Pinto Dos Santos D, Stanzione A, Ugga L, Zwanenburg A. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging 2023; 14:75. [PMID: 37142815 PMCID: PMC10160267 DOI: 10.1186/s13244-023-01415-8] [Citation(s) in RCA: 91] [Impact Index Per Article: 91.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 03/24/2023] [Indexed: 05/06/2023] Open
Abstract
Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.
Collapse
Affiliation(s)
- Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey.
| | - Bettina Baessler
- Institute of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Spyridon Bakas
- Center for Artificial Intelligence for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing & Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Renato Cuocolo
- Department of Medicine, Surgery, and Dentistry, University of Salerno, Baronissi, Italy
| | - Andrey Fedorov
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Nathaniel Mercaldo
- Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA, USA
- Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Henning Müller
- University of Applied Sciences of Western Switzerland (HES-SO Valais), Valais, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva (UniGe), Geneva, Switzerland
| | - Fanny Orlhac
- Laboratoire d'Imagerie Translationnelle en Oncologie (LITO)-U1288, Institut Curie, Inserm, Université PSL, Orsay, France
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Institute for Diagnostic and Interventional Radiology, Goethe-University Frankfurt Am Main, Frankfurt, Germany
| | - Arnaldo Stanzione
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Lorenzo Ugga
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Alex Zwanenburg
- OncoRay-National Center for Radiation Research in Oncology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany
- National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany
- German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
33
|
Arbogast KB, Funk JR, Solomon G, Crandall J. Measuring Head Acceleration Like a CHAMP. J Athl Train 2023; 58:283-284. [PMID: 36521167 PMCID: PMC11215641 DOI: 10.4085/1062-6050-0516.22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/20/2023]
Affiliation(s)
- Kristy B. Arbogast
- Center for Injury Research and Prevention, Children's Hospital of Philadelphia, and Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia
| | | | - Gary Solomon
- Player Health and Safety Department, National Football League, New York, NY
| | | |
Collapse
|
34
|
Zywicka EM, Elliott L, Twine CP, Mouton R, Hinchliffe RJ. Protocol for a systematic review of reporting standards of lower limb endovascular interventions in peripheral arterial disease. Syst Rev 2023; 12:20. [PMID: 36788631 PMCID: PMC9930275 DOI: 10.1186/s13643-023-02182-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 02/02/2023] [Indexed: 02/16/2023] Open
Abstract
INTRODUCTION Techniques and devices for the endovascular treatment of peripheral arterial disease (PAD) are continuously evolving. High-quality clinical trials limit the variation in how endovascular interventions are described, performed and reported. The aim of this systematic review is to assess the quality of reporting standards in randomised controlled trials (RCTs) of endovascular lower limb interventions against the Consolidated Standards of Reporting Trials for Non-Pharmacologic Treatments (CONSORT-NPT) and template for intervention description and replication (TIDieR) framework. METHODS Randomised trials including participants with peripheral arterial disease undergoing any infra-inguinal lower limb endovascular arterial intervention, searched from Medline, Embase and Cochrane Library databases from inception to December 2021, will be included. All study data, including details of the procedure investigated, will be extracted in keeping with the CONSORT-NPT and TIDieR checklist. Descriptive statistics will be used to summarise general study details and reporting standards of the trials. DISCUSSION The results will be used to inform the design of future RCTs in this area by optimising the way the interventions are described, standardised, and monitored. The systematic review will be disseminated via peer-reviewed manuscripts and presentations at relevant conferences. SYSTEMATIC REVIEW REGISTRATION PROSPERO CRD42022288214.
Collapse
Affiliation(s)
- Ewa M Zywicka
- Translational Health Sciences, University of Bristol Medical School, Bristol, UK. .,Southmead Hospital, North Bristol NHS Trust, Bristol, UK.
| | - Lucy Elliott
- Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| | - Christopher P Twine
- Translational Health Sciences, University of Bristol Medical School, Bristol, UK.,Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| | - Ronelle Mouton
- Translational Health Sciences, University of Bristol Medical School, Bristol, UK.,Southmead Hospital, North Bristol NHS Trust, Bristol, UK
| | - Robert J Hinchliffe
- Translational Health Sciences, University of Bristol Medical School, Bristol, UK.,Southmead Hospital, North Bristol NHS Trust, Bristol, UK.,Centre for Surgical Research, University of Bristol Medical School, Bristol, UK
| |
Collapse
|
35
|
'Small Data' for big insights in ecology. Trends Ecol Evol 2023:S0169-5347(23)00019-8. [PMID: 36797167 DOI: 10.1016/j.tree.2023.01.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 01/18/2023] [Accepted: 01/25/2023] [Indexed: 02/17/2023]
Abstract
Big Data science has significantly furthered our understanding of complex systems by harnessing large volumes of data, generated at high velocity and in great variety. However, there is a risk that Big Data collection is prioritised to the detriment of 'Small Data' (data with few observations). This poses a particular risk to ecology where Small Data abounds. Machine learning experts are increasingly looking to Small Data to drive the next generation of innovation, leading to development in methods for Small Data such as transfer learning, knowledge graphs, and synthetic data. Meanwhile, meta-analysis and causal reasoning approaches are evolving to provide new insights from Small Data. These advances should add value to high-quality Small Data catalysing future insights for ecology.
Collapse
|
36
|
Audet S, Doyle C, Lemieux C, Tardif MA, Gauvreau A, Simonyan D, Nabi H, Lemieux J. Adherence to CONSORT Guidelines and Reporting of the Determinants of External Validity in Clinical Oncology Randomized Controlled Trials: A Review of Trials Published in Four Major Journals between 2013 and 2015. Curr Oncol 2023; 30:2061-2072. [PMID: 36826121 PMCID: PMC9955486 DOI: 10.3390/curroncol30020160] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 01/18/2023] [Accepted: 01/26/2023] [Indexed: 02/10/2023] Open
Abstract
Our primary objective was to determine the proportion of trials that report the number of patients assessed for eligibility before randomization. We performed the systematic retrieval and analysis of all phase II, III, and IV RCTs published between 2013 and 2015 in four high-impact-factor journals in the field of clinical oncology. Among 456 RCTs reviewed, 236 trials (51.8%) reported the number of patients assessed for eligibility. Among the 236 trials that reported the entire enrollment process, the reasons for patient exclusion could be found in 184 trials (78%). A flow diagram was presented in 452 trials (99.1%), and 98 trials (21.5%) included a discussion on generalizability. Reporting the parameters of external validity in medical oncology RCTs is challenging. Improving adherence to the 2010 CONSORT guidelines concerning the enrollment process could help clinicians and health policymakers establish to whom trial results apply.
Collapse
Affiliation(s)
- Sophie Audet
- Département d’Hématologie, Hôpital de Chicoutimi, Chicoutimi, QC G7H 5H6, Canada
| | - Catherine Doyle
- Département d’Hématologie, Centre Hospitalier Universitaire (CHU) de Québec—Université Laval, Québec, QC G1R 2J6, Canada
- Faculté de Médecine, Université Laval, Québec, QC G1V 0A6, Canada
- Centre de Recherche du CHU de Québec—Université Laval, Québec, QC G1V 4G2, Canada
| | - Christopher Lemieux
- Département d’Hématologie, Centre Hospitalier Universitaire (CHU) de Québec—Université Laval, Québec, QC G1R 2J6, Canada
- Faculté de Médecine, Université Laval, Québec, QC G1V 0A6, Canada
- Centre de Recherche du CHU de Québec—Université Laval, Québec, QC G1V 4G2, Canada
| | - Marc-Antoine Tardif
- Département d’Hémato-oncologue, Hôpital de Rimouski, Rimouski, QC G5L 5T1, Canada
| | - Andréa Gauvreau
- Faculté de Médecine, Université Laval, Québec, QC G1V 0A6, Canada
| | - David Simonyan
- Centre de Recherche du CHU de Québec—Université Laval, Québec, QC G1V 4G2, Canada
| | - Hermann Nabi
- Centre de Recherche du CHU de Québec—Université Laval, Québec, QC G1V 4G2, Canada
| | - Julie Lemieux
- Département d’Hématologie, Centre Hospitalier Universitaire (CHU) de Québec—Université Laval, Québec, QC G1R 2J6, Canada
- Faculté de Médecine, Université Laval, Québec, QC G1V 0A6, Canada
- Centre de Recherche du CHU de Québec—Université Laval, Québec, QC G1V 4G2, Canada
- Correspondence:
| |
Collapse
|
37
|
Cho VY, Loh XH, Abbott L, Mohd-Isa NA, Anthonappa RP. Reporting Eye-tracking Studies In DEntistry (RESIDE) checklist. J Dent 2023; 129:104359. [PMID: 36403692 DOI: 10.1016/j.jdent.2022.104359] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 10/18/2022] [Accepted: 11/05/2022] [Indexed: 11/18/2022] Open
Abstract
OBJECTIVES To (i) provide a scoping review of eye-tracking studies in dentistry, and (ii) propose a "Reporting Eye-tracking Studies in DEntistry" (RESIDE) checklist to facilitate standard reporting of eye-tracking studies. DATA A comprehensive search of six distinct electronic databases was undertaken. SOURCES Pubmed, OVID, Wed of Knowledge, Scopus, Cochrane and Google Scholar were used to identify studies that employed eye-tracking technology and dentistry as a subfield STUDY SELECTION: 42 studies met the inclusion criteria. Most studies exhibited several inconsistencies or failed to report on the appropriate items in the RESIDE checklist. These essential components include ethical approval, sample size calculation, location and setting, eye-tracking device attributes, participant calibration, sequence of events, and eye-tracking metrics (quantitative, qualitative and data details). CONCLUSIONS Evaluation of the published eye-tracking studies in this scoping review provides empirical data, highlighting the inconsistencies and limitations. Importantly, it illustrates the applicability of the RESIDE Checklist, which provides a comprehensive list of reporting elements to assist authors and reviewers of eye-tracking studies in dentistry. Also, RESIDE provides a framework to overcome critical issues to ensure high-quality scientific publications. CLINICAL SIGNIFICANCE A minimum threshold should be applied before accepting eye-tracking studies for publication in the future. RESIDE checklist promotes transparent and reproducible scientific communication about eye-tracking applications to dentistry. In addition, it provides a comprehensive list of reporting elements to assist authors and reviewers in ensuring high-quality scientific publications.
Collapse
Affiliation(s)
- Vanessa Y Cho
- Dental School, The University of Western Australia, 17 Monash Ave, Nedlands 6009, Perth, Australia.
| | - Xin Hui Loh
- Dental School, The University of Western Australia, 17 Monash Ave, Nedlands 6009, Perth, Australia.
| | - Lyndon Abbott
- Dental School, The University of Western Australia, 17 Monash Ave, Nedlands 6009, Perth, Australia.
| | - Nur Anisah Mohd-Isa
- Dental School, The University of Western Australia, 17 Monash Ave, Nedlands 6009, Perth, Australia.
| | - Robert P Anthonappa
- Dental School, The University of Western Australia, 17 Monash Ave, Nedlands 6009, Perth, Australia.
| |
Collapse
|
38
|
Wilson E, Cruz F, Maclean D, Ghanawi J, McCann S, Brennan P, Liao J, Sena E, Macleod M. Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier. Clin Sci (Lond) 2023; 137:181-193. [PMID: 36630537 PMCID: PMC9885807 DOI: 10.1042/cs20220594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 12/15/2022] [Accepted: 01/11/2023] [Indexed: 01/13/2023]
Abstract
OBJECTIVE Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records. METHODS We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies. RESULTS Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700. CONCLUSION In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.
Collapse
Affiliation(s)
- Emma Wilson
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
- Correspondence: Emma Wilson ()
| | - Florenz Cruz
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, QUEST Center, Berlin, Germany
| | - Duncan Maclean
- University of Edinburgh Medical School, University of Edinburgh, Edinburgh, U.K
| | | | - Sarah K. McCann
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, QUEST Center, Berlin, Germany
| | - Paul M. Brennan
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Jing Liao
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| |
Collapse
|
39
|
Zhang XJ, Wang YJ, Lu X, Ying PJ, Qian SY, Liang J, Zheng GQ. The quality of reporting of randomized controlled trials of HuatuoZaizao pill for stroke. Front Pharmacol 2023; 13:1106957. [PMID: 36703753 PMCID: PMC9871376 DOI: 10.3389/fphar.2022.1106957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Accepted: 12/19/2022] [Indexed: 01/12/2023] Open
Abstract
Background: HuatuoZaizao pill (HZP), a Chinese patent medicine, is often used in the treatment of stroke. However, there is still a lack of enough evidence to recommend the routine use of HZP for stroke. This study is aimed at evaluating the quality of reporting of randomized controlled trials (RCTs) on HZP for stroke. Methods: RCTs on HuatuoZaizao pill for stroke were evaluated by using Consolidated Standards of Reporting Trials (CONSORT) guidelines and CONSORT extension criteria on reporting herbal interventions (CONSORT-CHM) guidelines. Microsoft Excel 2007 and SPSS20.0 was used for statistics analyses. Results: Seventeen studies involving 1801 stroke patients were identified. CONSORT-CHM has expanded 24.3% (9/37) items in CONSORT and added a small item. The average scores of CONSORT evaluation is 14.6, while the average scores of the CONSORT-CHM evaluation is 11.6. The central items in CONSORT as eligibility criterion, sample size calculation, primary outcome, method of randomization sequence generation, allocation concealment, implementation of randomization, description of blinding, and detailed statistical methods were reported in 77%, 6%, 100%, 47%, 6%, 6%, 6%, and 94% of trials, respectively. In terms of the CONSORT-CHM, none of the articles reported in detail the dosage form, origin, formula basis and so on of HZP, and only half of studies reported the outcome indicators related to Traditional Chinese Medicine syndromes. Conclusion: The overall report quality of RCT related to HZP is low. HZP still needs to report higher quality RCTs to prove its effectiveness and safety.
Collapse
Affiliation(s)
- Xiao-Jie Zhang
- Department of Neurology, The Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou, China
| | - Yi-Jing Wang
- Department of Neurology, The Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou, China
| | - Xiao Lu
- Department of Neurology, The Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou, China
| | - Peng-Jie Ying
- Department of Neurology, The Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou, China
| | - Shi-Yan Qian
- Department of Neurology, The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
| | - Jie Liang
- Department of Neurology, The Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou, China
| | - Guo-Qing Zheng
- Department of Neurology, The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China,*Correspondence: Guo-Qing Zheng,
| |
Collapse
|
40
|
ALARCÓN MA, ARIZA-FREITAS T, CHAVEZ-VEREAU N, LÓPEZ-PACHECO A, PANNUTI CM, MÁLAGA-FIGUEROA L. Consistency of recommendations of clinical practice guidelines in periodontology: a systematic review. Braz Oral Res 2023; 37:e029. [PMID: 37018810 DOI: 10.1590/1807-3107bor-2023.vol37.0029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 09/19/2022] [Indexed: 04/05/2023] Open
Abstract
The aim of this systematic review was to evaluate the methodological quality and the consistency of recommendations of clinical practice guidelines (CPGs) in Periodontology. An electronic search was conducted in two databases, MEDLINE and EMBASE, eight CPGs databases, and home pages of scientific societies in Periodontology up to April 2022. Three reviewers independently assessed methodological quality using the AGREE II instrument. In addition, we evaluated the consistency of the recommendations. Eleven CPGs were included, and the topics developed focused on prevention, diagnosis, risk factors, surgical and non-surgical periodontal treatment, antimicrobial therapy, root coverage, and maintenance. We found that the AGREE domains 2 (Stakeholder involvement) and 5 (Applicability) obtained the lowest scores. Domains 1 (Scope and purpose), 3 (Rigor of development) and 4 (Clarity of presentation) obtained the highest scores among the evaluated CPGs. The clinical recommendations for treatment of periodontal diseases were mostly consistent. Overall, the quality of CPGs used in periodontics was high. There was consistency of recommendations in specific fields. These findings may help researchers to promote CPGs focused on different fields of periodontics that have not yet been developed. Furthermore, the clinician will be able to make better clinical decisions.
Collapse
|
41
|
Towards a new paradigm for ‘journal quality’ criteria: a scoping review. Scientometrics 2022. [DOI: 10.1007/s11192-022-04520-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
42
|
Sweetnam C, Goulding L, Davis RE, Khadjesari Z, Boaz A, Healey A, Sevdalis N, Bakolis I, Hull L. Development and psychometric evaluation of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool: a study protocol. BMJ Open 2022; 12:e061209. [PMID: 36526311 PMCID: PMC9764655 DOI: 10.1136/bmjopen-2022-061209] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Accepted: 09/29/2022] [Indexed: 12/23/2022] Open
Abstract
INTRODUCTION The need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research. METHODS AND ANALYSIS This study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published in Implementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field. ETHICS AND DISSEMINATION This study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King's College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).
Collapse
Affiliation(s)
- Chloe Sweetnam
- Neurology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Lucy Goulding
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
| | - Rachel E Davis
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
| | - Zarnie Khadjesari
- Behavioural and Implementation Science Research Group, School of Health Sciences, University of East Anglia, Norwich, UK
| | - Annette Boaz
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK
| | - Andy Healey
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
- King's Health Economics, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Nick Sevdalis
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
| | - Ioannis Bakolis
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Louise Hull
- Centre for Implementation Science, Health Service and Population Research Department, King's College London, London, UK
| |
Collapse
|
43
|
Ravichandran C, Babb SM, Ongur D, Harris PQ, Cohen BM. Survey of quality and clarity of methods and results reporting in 1 year of intervention studies published in high-impact medical and psychiatric journals. BMJ Open 2022; 12:e061882. [PMID: 36523238 PMCID: PMC9748970 DOI: 10.1136/bmjopen-2022-061882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
OBJECTIVE We assessed how well articles in major medical and psychiatric journals followed best reporting practices in presenting results of intervention studies. METHOD Standardised data collection was used to review studies in high-impact and widely read medical (JAMA, Lancet and New England Journal of Medicine) and psychiatric (American Journal of Psychiatry, JAMA Psychiatry, Journal of Clinical Psychiatry and Lancet Psychiatry) journals, published between 1 September 2018 and 31 August 2019. Two team members independently reviewed each article. MEASURES The primary outcome measure was proportion of papers reporting consensus elements required to understand and evaluate the results of the intervention. The secondary outcome measure was comparison of complete and accessible reporting in the major medical versus the major psychiatric journals. RESULTS One hundred twenty-seven articles were identified for inclusion. At least 90% of articles in both medical and psychiatric journals included sample size, statistical significance, randomisation method, elements of study flow, and age, sex, and illness severity by randomisation group. Selected elements less frequently reported by either journal type were confidence intervals in the abstract, reported in 93% (95% CI 84% to 97%) of medical journal articles and 58% (95% CI 45% to 69%) of psychiatric journal articles, and sample size method (93%, 95% CI 84% to 97% medical; 69%, 95% CI 57% to 80% psychiatric), race and ethnicity by randomisation group (51%, 95% CI 40% to 63% medical; 73%, 95% CI 60% to 83% psychiatric), and adverse events (94%; 95% CI 86% to 98% medical; 80%, 95% CI 68% to 88% psychiatric) in the main text. CIs were included less often in psychiatric than medical journals (p<0.004 abstract, p=0.04 main text, after multiple-testing correction). CONCLUSIONS Recommendations include standard inclusion of a table specifying the outcome(s) designated as primary, and the sample size, effect size(s), CI(s) and p value(s) corresponding to the primary test(s) for efficacy.
Collapse
Affiliation(s)
- Caitlin Ravichandran
- Psychiatry, McLean Hospital, Belmont, Massachusetts, USA
- Psychiatry, Lurie Center for Autism, Lexington, Massachusetts, USA
- Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
| | - Suzann M Babb
- Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
- Program for Neuropsychiatric Research, McLean Hospital, Belmont, Massachusetts, USA
| | - Dost Ongur
- Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
- Schizophrenia and Bipolar Disorder Research Program, McLean Hospital, Belmont, Massachusetts, USA
| | - Peter Q Harris
- Department of Psychiatry, Jersey Shore University Medical Center, Neptune City, New Jersey, USA
- Psychiatry, Aetna, Hartford, Connecticut, USA
| | - Bruce M Cohen
- Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
- Program for Neuropsychiatric Research, McLean Hospital, Belmont, Massachusetts, USA
| |
Collapse
|
44
|
Hansford HJ, Cashin AG, Bagg MK, Wewege MA, Ferraro MC, Kianersi S, Mayo-Wilson E, Grant SP, Toomey E, Skinner IW, McAuley JH, Lee H, Jones MD. Feasibility of an Audit and Feedback Intervention to Facilitate Journal Policy Change Towards Greater Promotion of Transparency and Openness in Sports Science Research. SPORTS MEDICINE - OPEN 2022; 8:101. [PMID: 35932429 PMCID: PMC9357245 DOI: 10.1186/s40798-022-00496-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Accepted: 07/21/2022] [Indexed: 11/10/2022]
Abstract
Objectives To evaluate (1) the feasibility of an audit-feedback intervention to facilitate sports science journal policy change, (2) the reliability of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) policy evaluation form, and (3) the extent to which policies of sports science journals support transparent and open research practices. Methods We conducted a cross-sectional, audit-feedback, feasibility study of transparency and openness standards of the top 38 sports science journals by impact factor. The TRUST form was used to evaluate journal policies support for transparent and open research practices. Feedback was provided to journal editors in the format of a tailored letter. Inter-rater reliability and agreement of the TRUST form was assessed using intraclass correlation coefficients and the standard error of measurement, respectively. Time-based criteria, fidelity of intervention delivery and qualitative feedback were used to determine feasibility. Results The audit-feedback intervention was feasible based on the time taken to rate journals and provide tailored feedback. The mean (SD) score on the TRUST form (range 0–27) was 2.05 (1.99), reflecting low engagement with transparent and open practices. Inter-rater reliability of the overall score of the TRUST form was moderate [ICC (2,1) = 0.68 (95% CI 0.55–0.79)], with standard error of measurement of 1.17. However, some individual items had poor reliability. Conclusion Policies of the top 38 sports science journals have potential for improved support for transparent and open research practices. The feasible audit-feedback intervention developed here warrants large-scale evaluation as a means to facilitate change in journal policies. Registration: OSF (https://osf.io/d2t4s/).
Supplementary Information The online version contains supplementary material available at 10.1186/s40798-022-00496-x.
Collapse
|
45
|
Rowe A. Recommendations to improve use and reporting of statistics in animal experiments. Lab Anim 2022:236772221140669. [DOI: 10.1177/00236772221140669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
Abstract
Flaws in experimental statistics are a major contributor to the poor reproducibility of animal experiments. Informed decisions about whether conclusions are justified requires clear reporting of experimental data and the statistical methods used to analyse them. When data are misinterpreted, manipulated or concealed to generate publications, it creates an illusion that chance observations are robust data which confirm the hypotheses presented. Attempts to reproduce and advance such observations can propagate large areas of irreproducible science. This hinders scientific progress, erodes public support for research, damages reputations and wastes resources. This review analyses and explains recommendations to improve use and reporting of statistics in animal experiments.
Collapse
|
46
|
Niederberger M, Deckert S. [The Delphi technique: Methodology, variants and usage examples]. ZEITSCHRIFT FUR EVIDENZ, FORTBILDUNG UND QUALITAT IM GESUNDHEITSWESEN 2022; 174:11-19. [PMID: 36137932 DOI: 10.1016/j.zefq.2022.08.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Revised: 05/20/2022] [Accepted: 08/23/2022] [Indexed: 12/14/2022]
Abstract
In the field of medicine and health sciences, Delphi methods are applied mainly in the exploratory or evaluative phases of a research process. Explicit and implicit knowledge of respected experts from research and practice is systematically synthesized. Originally developed as a method for structuring a group communication process, Delphi techniques have been established in the health sector as a consensus method. The findings are used to improve the evidence and acceptance of planned interventions or necessary standards or guidelines and to increase the probability of successful implementation in practice. However, different variants of Delphi methods have been developed in recent years, which are systematically contrasted and reflected in this paper with regard to key epistemological and methodological research activities. Based on this overview, researchers should be enabled to select the most suitable Delphi technique for their own research questions and research endeavors.
Collapse
Affiliation(s)
- Marlen Niederberger
- Forschungsmethoden in der Gesundheitsförderung und Prävention, Pädagogische Hochschule Schwäbisch Gmünd, Schwäbisch Gmuünd, Deutschland.
| | - Stefanie Deckert
- Zentrum für Evidenzbasierte Gesundheitsversorgung, Universitätsklinikum und Medizinische Fakultät Carl Gustav Carus an der Technischen Universität Dresden, Dresden, Deutschland
| |
Collapse
|
47
|
Nunamaker EA, Reynolds PS. 'Invisible actors'-How poor methodology reporting compromises mouse models of oncology: A cross-sectional survey. PLoS One 2022; 17:e0274738. [PMID: 36264974 PMCID: PMC9584398 DOI: 10.1371/journal.pone.0274738] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Accepted: 08/28/2022] [Indexed: 11/05/2022] Open
Abstract
The laboratory mouse is a key player in preclinical oncology research. However, emphasis of techniques reporting at the expense of critical animal-related detail compromises research integrity, animal welfare, and, ultimately, the translation potential of mouse-based oncology models. To evaluate current reporting practices, we performed a cross-sectional survey of 400 preclinical oncology studies using mouse solid-tumour models. Articles published in 2020 were selected from 20 journals that specifically endorsed the ARRIVE (Animal Research: Reporting of In Vivo Experiments) preclinical reporting guidelines. We assessed reporting compliance for 22 items in five domains: ethical oversight assurance, animal signalment, husbandry, welfare, and euthanasia. Data were analysed using hierarchical generalised random-intercept models, clustered on journal. Overall, reporting of animal-related items was poor. Median compliance over all categories was 23%. There was little or no association between extent of reporting compliance and journal or journal impact factor. Age, sex, and source were reported most frequently, but verifiable strain information was reported for <10% of studies. Animal husbandry, housing environment, and welfare items were reported by <5% of studies. Fewer than one in four studies reported analgesia use, humane endpoints, or an identifiable method of euthanasia. Of concern was the poor documentation of ethical oversight information. Fewer than one in four provided verifiable approval information, and almost one in ten reported no information, or information that was demonstrably false. Mice are the "invisible actors" in preclinical oncology research. In spite of widespread endorsement of reporting guidelines, adherence to reporting guidelines on the part of authors is poor and journals fail to enforce guideline reporting standards. In particular, the inadequate reporting of key animal-related items severely restricts the utility and translation potential of mouse models, and results in research waste. Both investigators and journals have the ethical responsibility to ensure animals are not wasted in uninformative research.
Collapse
Affiliation(s)
- Elizabeth A. Nunamaker
- Animal Care Services, University of Florida, Gainesville, Florida, United States of America
| | - Penny S. Reynolds
- Department of Anesthesiology, Statistics in Anesthesiology Research (STAR) Core, College of Medicine, University of Florida, Gainesville, Florida, United States of America
- * E-mail:
| |
Collapse
|
48
|
Murphy RP, O'Donnell MJ, Nolan A, McGrath E, O'Conghaile A, Ferguson J, Alvarez-Iglesias A, Costello M, Loughlin E, Reddin C, Ruttledge S, Gorey S, Hughes D, Smyth A, Canavan M, Judge C. Effect of a Run-In Period on Estimated Treatment Effects in Cardiovascular Randomized Clinical Trials: A Meta-Analytic Review. J Am Heart Assoc 2022; 11:e023061. [PMID: 36250666 DOI: 10.1161/jaha.121.023061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background A run-in period may increase adherence to intervention and reduce loss to follow-up. Whether use of a run-in period affects the magnitude of treatment effects is unknown. Methods and Results We conducted a meta-analysis comparing treatment effects from 11 systematic reviews of cardiovascular prevention trials using a run-in period with matched trials not using a run-in period. We matched run-in with non-run-in trials by population, intervention, control, and outcome. We calculated a ratio of relative risks (RRRs) using a random-effects meta-analysis. Our primary outcome was a composite of cardiovascular events, and the primary analysis was a matched comparison of clinical trials using a run-in period versus without a run-in period. We identified 66 run-in trials and 111 non-run-in trials (n=668 901). On meta-analysis there was no statistically significant difference in the magnitude of treatment effect between run-in trials (relative risk [RR], 0.83 [95% CI, 0.80-0.87]) compared with non-run-in trials (RR, 0.88 [95% CI, 0.84-0.91]; RRR, 0.95 [95% CI, 0.90-1.01]). There was no significant difference in the RRR for secondary outcomes of all-cause mortality (RRR, 0.97 [95% CI, 0.91-1.03]) or medication discontinuation because of adverse events (RRR, 1.05 [95% CI, 0.85-1.21]). Post hoc exploratory univariate meta-regression showed that on average a run-in period is associated with a statistically significant difference in treatment effect (RRR, 0.94 [95% CI, 0.90-0.99]) for cardiovascular composite outcome, but this was not statistically significant on multivariable meta-regression analysis (RRR, 0.95 [95% CI, 0.90-1.0]). Conclusions The use of a run-in period was not associated with a difference in the magnitude of treatment effect among cardiovascular prevention trials.
Collapse
Affiliation(s)
- Robert P Murphy
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Martin J O'Donnell
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Aoife Nolan
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Emer McGrath
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland.,Department of Neurology Harvard Medical School Boston MA
| | - Aengus O'Conghaile
- Department of Psychiatry National University of Ireland Galway Galway Ireland
| | - John Ferguson
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Alberto Alvarez-Iglesias
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Maria Costello
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Elaine Loughlin
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Catriona Reddin
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Sarah Ruttledge
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Sarah Gorey
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Diarmaid Hughes
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Andrew Smyth
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Michelle Canavan
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland
| | - Conor Judge
- Health Research Board Clinical Research Facility-Galway National University of Ireland Galway Galway Ireland.,Translational Medical Device Lab National University of Ireland Galway Galway Ireland.,Wellcome Trust - Health Research Board Irish Clinical Academic Training Galway Ireland
| |
Collapse
|
49
|
Methodological and ethical quality of surgical trials from 2016 to 2020. Langenbecks Arch Surg 2022; 407:3793-3802. [PMID: 36029311 DOI: 10.1007/s00423-022-02649-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2022] [Accepted: 08/04/2022] [Indexed: 10/15/2022]
Abstract
PURPOSE Randomized controlled trials (RCTs) are the gold standard tool used to evaluate therapeutic interventions. The published results showed that progress still needs to be made not only from a methodological point of view but also from an ethical point of view. The aim of this study was to evaluate the methodological and ethical qualities of randomized clinical trials in surgery over the last few years. METHODS All of the articles chosen for review reported on randomized controlled surgical trials and were published in 10 international journals between 2016 and 2020. Eligible studies were identified, selected, and then evaluated based on a broad set of predetermined criteria. Methodological quality was evaluated using the Jadad scale, and ethical quality was evaluated using the Berdeu score. RESULTS The methodological quality score (Jadad scale) ranged from 5 to 13, with a mean of 10.0 ± 1.54. The methodological quality was insufficient (score ≤ 9) for 162 trials (31.2%). The ethical quality score ranged from 0.25 to 1, with a mean of 0.8 ± 0.11. Fifty-two articles (10%) did not state that informed consent was requested from the participants, and 21 articles (4%) did not report either research ethics committee or institutional committee protocol approval. CONCLUSION The randomized clinical surgical trials analyzed showed that they had satisfactory methods in only 70% of the cases and that they had respected the fundamental ethical principles in 90% of the cases. However, less than 8% of the studies reported planned interim analysis, prospectively defined stopping rules, and independent monitoring committee.
Collapse
|
50
|
Arbogast KB, Caccese JB, Buckley TA, McIntosh AS, Henderson K, Stemper BD, Solomon G, Broglio SP, Funk JR, Crandall JR. Consensus Head Acceleration Measurement Practices (CHAMP): Origins, Methods, Transparency and Disclosure. Ann Biomed Eng 2022; 50:1317-1345. [PMID: 35920964 PMCID: PMC9652170 DOI: 10.1007/s10439-022-03025-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 07/13/2022] [Indexed: 11/30/2022]
Abstract
The use of head kinematic measurement devices has recently proliferated owing to technology advances that make such measurement more feasible. In parallel, demand to understand the biomechanics of head impacts and injury in sports and the military has increased as the burden of such loading on the brain has received focused attention. As a result, the field has matured to the point of needing methodological guidelines to improve the rigor and consistency of research and reduce the risk of scientific bias. To this end, a diverse group of scientists undertook a comprehensive effort to define current best practices in head kinematic measurement, culminating in a series of manuscripts outlining consensus methodologies and companion summary statements. Summary statements were discussed, revised, and voted upon at the Consensus Head Acceleration Measurement Practices (CHAMP) Conference in March 2022. This manuscript summarizes the motivation and methods of the consensus process and introduces recommended reporting checklists to be used to increase transparency and rigor of future experimental design and publication of work in this field. The checklists provide an accessible means for researchers to apply the best practices summarized in the companion manuscripts when reporting studies utilizing head kinematic measurement in sport and military settings.
Collapse
Affiliation(s)
- Kristy B Arbogast
- Center for Injury Research and Prevention, Children's Hospital of Philadelphia, 2716 South St., PA, 19146, Philadelphia, USA. .,Department of Pediatrics, University of Pennsylvania, PA, Philadelphia, USA.
| | - Jaclyn B Caccese
- The Ohio State University Chronic Brain Injury Program, The Ohio State University College of Medicine, Columbus, OH, USA
| | - Thomas A Buckley
- Department of Kinesiology and Applied Physiology, University of Delaware, Newark, DE, USA
| | - Andrew S McIntosh
- McIntosh Consultancy and Research, Sydney, NSW, Australia.,Monash University Accident Research Centre, Monash University, Melbourne, VIC, Australia.,School of Engineering, Edith Cowan University, Perth, WA, Australia
| | | | - Brian D Stemper
- Joint Department of Biomedical Engineering, Medical College of Wisconsin & Marquette University, Milwaukee, WI, USA.,Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA.,Neuroscience Research, Zablocki Veterans Affairs Medical Center, Milwaukee, WI, USA
| | - Gary Solomon
- National Football League Player Health and Safety, New York, NY, USA
| | - Steven P Broglio
- Michigan Concussion Center, University of Michigan, Ann Arbor, MI, USA
| | | | | |
Collapse
|