1
|
Logullo P, van Zuuren EJ, Winchester CC, Tovey D, Gattrell WT, Price A, Harrison N, Goldman K, Chisholm A, Walters K, Blazey P. ACcurate COnsensus Reporting Document (ACCORD) explanation and elaboration: Guidance and examples to support reporting consensus methods. PLoS Med 2024; 21:e1004390. [PMID: 38709851 DOI: 10.1371/journal.pmed.1004390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/08/2024] Open
Abstract
BACKGROUND When research evidence is limited, inconsistent, or absent, healthcare decisions and policies need to be based on consensus among interested stakeholders. In these processes, the knowledge, experience, and expertise of health professionals, researchers, policymakers, and the public are systematically collected and synthesised to reach agreed clinical recommendations and/or priorities. However, despite the influence of consensus exercises, the methods used to achieve agreement are often poorly reported. The ACCORD (ACcurate COnsensus Reporting Document) guideline was developed to help report any consensus methods used in biomedical research, regardless of the health field, techniques used, or application. This explanatory document facilitates the use of the ACCORD checklist. METHODS AND FINDINGS This paper was built collaboratively based on classic and contemporary literature on consensus methods and publications reporting their use. For each ACCORD checklist item, this explanation and elaboration document unpacks the pieces of information that should be reported and provides a rationale on why it is essential to describe them in detail. Furthermore, this document offers a glossary of terms used in consensus exercises to clarify the meaning of common terms used across consensus methods, to promote uniformity, and to support understanding for consumers who read consensus statements, position statements, or clinical practice guidelines (CPGs). The items are followed by examples of reporting items from the ACCORD guideline, in text, tables, and figures. CONCLUSIONS The ACCORD materials-including the reporting guideline and this explanation and elaboration document-can be used by anyone reporting a consensus exercise used in the context of health research. As a reporting guideline, ACCORD helps researchers to be transparent about the materials, resources (both human and financial), and procedures used in their investigations so readers can judge the trustworthiness and applicability of their results/recommendations.
Collapse
Affiliation(s)
- Patricia Logullo
- Centre for Statistics in Medicine, University of Oxford, and EQUATOR Network UK Centre, Oxford, United Kingdom
| | | | - Christopher C Winchester
- Oxford PharmaGenesis, Oxford, United Kingdom
- Green Templeton College, University of Oxford, Oxford, United Kingdom
| | - David Tovey
- Journal of Clinical Epidemiology, London, United Kingdom
| | | | - Amy Price
- Dartmouth Institute for Health Policy & Clinical Practice (TDI), Geisel School of Medicine, Dartmouth College, Hanover, NH, USA, previously at Stanford Anesthesia, Informatics and Media Lab, Stanford University School of Medicine, Stanford, California, United States of America
| | | | - Keith Goldman
- Global Medical Affairs, AbbVie, North Chicago, Illinois, United States of America
| | | | | | - Paul Blazey
- Department of Medicine, University of British Columbia, Vancouver, Canada
| |
Collapse
|
2
|
Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, Ghassemi M, Liu X, Reitsma JB, van Smeden M, Boulesteix AL, Camaradou JC, Celi LA, Denaxas S, Denniston AK, Glocker B, Golub RM, Harvey H, Heinze G, Hoffman MM, Kengne AP, Lam E, Lee N, Loder EW, Maier-Hein L, Mateen BA, McCradden MD, Oakden-Rayner L, Ordish J, Parnell R, Rose S, Singh K, Wynants L, Logullo P. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024; 385:e078378. [PMID: 38626948 PMCID: PMC11019967 DOI: 10.1136/bmj-2023-078378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/17/2024] [Indexed: 04/19/2024]
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Karel G M Moons
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Science, Leiden University Medical Centre, Leiden, Netherlands
| | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Johannes B Reitsma
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Anne-Laure Boulesteix
- Department of Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-University of Munich, Munich, Germany
| | - Jennifer Catherine Camaradou
- Patient representative, Health Data Research UK patient and public involvement and engagement group
- Patient representative, University of East Anglia, Faculty of Health Sciences, Norwich Research Park, Norwich, UK
| | - Leo Anthony Celi
- Beth Israel Deaconess Medical Center, Boston, MA, USA
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, London, UK
- British Heart Foundation Data Science Centre, London, UK
| | - Alastair K Denniston
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Ben Glocker
- Department of Computing, Imperial College London, London, UK
| | - Robert M Golub
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Georg Heinze
- Section for Clinical Biometrics, Centre for Medical Data Science, Medical University of Vienna, Vienna, Austria
| | - Michael M Hoffman
- Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| | | | - Emily Lam
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Naomi Lee
- National Institute for Health and Care Excellence, London, UK
| | - Elizabeth W Loder
- The BMJ, London, UK
- Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Lena Maier-Hein
- Department of Intelligent Medical Systems, German Cancer Research Centre, Heidelberg, Germany
| | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK
- Wellcome Trust, London, UK
- Alan Turing Institute, London, UK
| | - Melissa D McCradden
- Department of Bioethics, Hospital for Sick Children Toronto, ON, Canada
- Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Johan Ordish
- Medicines and Healthcare products Regulatory Agency, London, UK
| | - Richard Parnell
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Sherri Rose
- Department of Health Policy and Center for Health Policy, Stanford University, Stanford, CA, USA
| | - Karandeep Singh
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Laure Wynants
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Patricia Logullo
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| |
Collapse
|
3
|
Collins GS, Whittle R, Bullock GS, Logullo P, Dhiman P, de Beyer JA, Riley RD, Schlussel MM. Open science practices need substantial improvement in prognostic model studies in oncology using machine learning. J Clin Epidemiol 2024; 165:111199. [PMID: 37898461 DOI: 10.1016/j.jclinepi.2023.10.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 10/06/2023] [Accepted: 10/20/2023] [Indexed: 10/30/2023]
Abstract
OBJECTIVE To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices. RESULTS We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs. CONCLUSION The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.
Collapse
Affiliation(s)
- Gary S Collins
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom.
| | - Rebecca Whittle
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
| | - Garrett S Bullock
- Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, NC, USA; Centre for Sport, Exercise and Osteoarthritis Research Versus Arthritis, University of Oxford, Oxford, United Kingdom
| | - Patricia Logullo
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
| | - Paula Dhiman
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
| | - Jennifer A de Beyer
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
| | - Michael M Schlussel
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
4
|
Innocenti T, Salvioli S, Logullo P, Giagio S, Ostelo R, Chiarotto A. The Uptake of the Core Outcome Set for Non-Specific Low Back Pain Clinical Trials is Poor: A Meta-Epidemiological Study of Trial Registrations. J Pain 2024; 25:31-38. [PMID: 37604361 DOI: 10.1016/j.jpain.2023.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 08/03/2023] [Accepted: 08/12/2023] [Indexed: 08/23/2023]
Abstract
We conducted a meta-epidemiological study on all non-specific low back pain (NSLBP) trial registrations on the World Health Organization International Clinical Trials Registry Platform and ClinicalTrials.gov. We aimed to 1) assess the uptake of the core outcome set (COS) for NSLBP in clinical trials; 2) assess the uptake of the core outcome measurement set for NSLBP in clinical trials; and 3) determine whether specific study characteristics are associated with the COS uptake. After applying the relevant filters for the condition, study type, and phase of the trial, 240 registry entries were included in this study. Only 50 (20.8%) entries showed a full COS uptake, and this rate did not increase over time. Most registry entries that planned to measure physical functioning (n = 152) used the Roland-Morris Disability Questionnaire (n = 74; 48.7%); a small percentage used the numeric rating scale (n = 60; 27.3%) or Short Form-12 (n = 5; 8.3%) if they planned to measure pain intensity (n = 220) or health-related quality of life (n = 60), respectively. Only the planned sample size (OR = 1.02; 95% CI = 1.01, 1.03) showed a significant but small association with COS uptake. The uptake of the COS for NSLBP is poor. Only 21% of the randomized controlled trials aimed to measure all COS domains in their study registration and COS uptake is not increased over time. Great heterogeneity in measurement instruments was also observed, revealing poor core outcome measurement set uptake. PERSPECTIVE: The Core Outcome Set (COS) for non-specific low back pain was published more than 20 years ago. We evaluated whether trial registrations are using this set of outcomes when testing interventions for low back pain. Full uptake was found only in 21% of the sample, and this is not increasing over time. Researchers should use the COS to ensure that trials measure relevant outcomes consistently.
Collapse
Affiliation(s)
- Tiziano Innocenti
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, the Netherlands; GIMBE Foundation, Bologna, Italy
| | - Stefano Salvioli
- GIMBE Foundation, Bologna, Italy; Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health, University of Genoa, Genoa, Italy
| | - Patricia Logullo
- Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases (NDORMS), University of Oxford, Oxford, United Kingdom
| | - Silvia Giagio
- Division of Occupational Medicine, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Biomedical and Neuromotor Sciences (DIBINEM), Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | - Raymond Ostelo
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, the Netherlands; Department of Epidemiology and Data Science, Amsterdam UMC, Location Vrije Universiteit, Amsterdam Movement Sciences research institute, the Netherlands
| | - Alessandro Chiarotto
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, the Netherlands; Department of General Practice, Erasmus MC, University Medical Center, Rotterdam, the Netherlands
| |
Collapse
|
5
|
Gattrell WT, Logullo P, van Zuuren EJ, Price A, Hughes EL, Blazey P, Winchester CC, Tovey D, Goldman K, Hungin AP, Harrison N. ACCORD (ACcurate COnsensus Reporting Document): A reporting guideline for consensus methods in biomedicine developed via a modified Delphi. PLoS Med 2024; 21:e1004326. [PMID: 38261576 PMCID: PMC10805282 DOI: 10.1371/journal.pmed.1004326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/25/2024] Open
Abstract
BACKGROUND In biomedical research, it is often desirable to seek consensus among individuals who have differing perspectives and experience. This is important when evidence is emerging, inconsistent, limited, or absent. Even when research evidence is abundant, clinical recommendations, policy decisions, and priority-setting may still require agreement from multiple, sometimes ideologically opposed parties. Despite their prominence and influence on key decisions, consensus methods are often poorly reported. Our aim was to develop the first reporting guideline dedicated to and applicable to all consensus methods used in biomedical research regardless of the objective of the consensus process, called ACCORD (ACcurate COnsensus Reporting Document). METHODS AND FINDINGS We followed methodology recommended by the EQUATOR Network for the development of reporting guidelines: a systematic review was followed by a Delphi process and meetings to finalize the ACCORD checklist. The preliminary checklist was drawn from the systematic review of existing literature on the quality of reporting of consensus methods and suggestions from the Steering Committee. A Delphi panel (n = 72) was recruited with representation from 6 continents and a broad range of experience, including clinical, research, policy, and patient perspectives. The 3 rounds of the Delphi process were completed by 58, 54, and 51 panelists. The preliminary checklist of 56 items was refined to a final checklist of 35 items relating to the article title (n = 1), introduction (n = 3), methods (n = 21), results (n = 5), discussion (n = 2), and other information (n = 3). CONCLUSIONS The ACCORD checklist is the first reporting guideline applicable to all consensus-based studies. It will support authors in writing accurate, detailed manuscripts, thereby improving the completeness and transparency of reporting and providing readers with clarity regarding the methods used to reach agreement. Furthermore, the checklist will make the rigor of the consensus methods used to guide the recommendations clear for readers. Reporting consensus studies with greater clarity and transparency may enhance trust in the recommendations made by consensus panels.
Collapse
Affiliation(s)
| | - Patricia Logullo
- Centre for Statistics in Medicine, University of Oxford, and EQUATOR Network UK Centre, Oxford, United Kingdom
| | | | - Amy Price
- Stanford Anesthesia, Informatics and Media Lab, Stanford University School of Medicine, Stanford, California, United States of America
| | | | - Paul Blazey
- Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Christopher C. Winchester
- Oxford PharmaGenesis, Oxford, United Kingdom
- Green Templeton College, University of Oxford, Oxford, United Kingdom
| | - David Tovey
- Journal of Clinical Epidemiology, London, United Kingdom
| | - Keith Goldman
- Global Medical Affairs, AbbVie, North Chicago, Illinois, United States of America
| | - Amrit Pali Hungin
- Faculty of Medical Sciences, Newcastle University, Newcastle, United Kingdom
| | | |
Collapse
|
6
|
Logullo P, de Beyer JA, Kirtley S, Schlüssel MM, Collins GS. Open access journal publication in health and medical research and open science: benefits, challenges and limitations. BMJ Evid Based Med 2023:bmjebm-2022-112126. [PMID: 37770125 DOI: 10.1136/bmjebm-2022-112126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/11/2023] [Indexed: 10/03/2023]
Affiliation(s)
- Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), NDORMS (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases), University of Oxford, Oxford, UK
- Oxford-Brazil EBM Alliance, Oxford, UK
| | - Jennifer A de Beyer
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), NDORMS (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases), University of Oxford, Oxford, UK
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), NDORMS (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases), University of Oxford, Oxford, UK
| | - Michael Maia Schlüssel
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), NDORMS (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases), University of Oxford, Oxford, UK
| | - Gary S Collins
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), NDORMS (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Diseases), University of Oxford, Oxford, UK
| |
Collapse
|
7
|
Schlussel MM, Sharp MK, de Beyer JA, Kirtley S, Logullo P, Dhiman P, MacCarthy A, Koroleva A, Speich B, Bullock GS, Moher D, Collins GS. Reporting guidelines used varying methodology to develop recommendations. J Clin Epidemiol 2023; 159:246-256. [PMID: 36965598 DOI: 10.1016/j.jclinepi.2023.03.018] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 03/13/2023] [Accepted: 03/21/2023] [Indexed: 03/27/2023]
Abstract
BACKGROUND AND OBJECTIVES We investigated the developing methods of reporting guidelines in the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network's database. METHODS In October 2018, we screened all records and excluded those not describing reporting guidelines from further investigation. Twelve researchers performed duplicate data extraction on bibliometrics, scope, development methods, presentation, and dissemination of all publications. Descriptive statistics were used to summarize the findings. RESULTS Of the 405 screened records, 262 described a reporting guidelines development. The number of reporting guidelines increased over the past 3 decades, from 5 in the 1990s and 63 in the 2000s to 157 in the 2010s. Development groups included 2-151 people. Literature appraisal was performed during the development of 56% of the reporting guidelines; 33% used surveys to gather external opinion on items to report; and 42% piloted or sought external feedback on their recommendations. Examples of good reporting for all reporting items were presented in 30% of the reporting guidelines. Eighteen percent of the reviewed publications included some level of spin. CONCLUSION Reporting guidelines have been developed with varying methodology. Reporting guideline developers should use existing guidance and take an evidence-based approach, rather than base their recommendations on expert opinion of limited groups of individuals.
Collapse
Affiliation(s)
- Michael M Schlussel
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK.
| | - Melissa K Sharp
- Health Research Board Centre for Primary Care Research, Department of General Practice, Royal College of Surgeons in Ireland, Dublin DO2 H638, Ireland
| | - Jennifer A de Beyer
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Paula Dhiman
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| | - Angela MacCarthy
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | | | - Benjamin Speich
- CLEAR Methos Center, Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Garrett S Bullock
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Gary S Collins
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
8
|
Logullo P, MacCarthy A, Dhiman P, Kirtley S, Ma J, Bullock G, Collins GS. Artificial intelligence in lung cancer diagnostic imaging: a review of the reporting and conduct of research published 2018-2019. BJR Open 2023; 5:20220033. [PMID: 37389003 PMCID: PMC10301715 DOI: 10.1259/bjro.20220033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Revised: 04/04/2023] [Accepted: 04/04/2023] [Indexed: 07/01/2023] Open
Abstract
Objective This study aimed to describe the methodologies used to develop and evaluate models that use artificial intelligence (AI) to analyse lung images in order to detect, segment (outline borders of), or classify pulmonary nodules as benign or malignant. Methods In October 2019, we systematically searched the literature for original studies published between 2018 and 2019 that described prediction models using AI to evaluate human pulmonary nodules on diagnostic chest images. Two evaluators independently extracted information from studies, such as study aims, sample size, AI type, patient characteristics, and performance. We summarised data descriptively. Results The review included 153 studies: 136 (89%) development-only studies, 12 (8%) development and validation, and 5 (3%) validation-only. CT scans were the most common type of image type used (83%), often acquired from public databases (58%). Eight studies (5%) compared model outputs with biopsy results. 41 studies (26.8%) reported patient characteristics. The models were based on different units of analysis, such as patients, images, nodules, or image slices or patches. Conclusion The methods used to develop and evaluate prediction models using AI to detect, segment, or classify pulmonary nodules in medical imaging vary, are poorly reported, and therefore difficult to evaluate. Transparent and complete reporting of methods, results and code would fill the gaps in information we observed in the study publications. Advances in knowledge We reviewed the methodology of AI models detecting nodules on lung images and found that the models were poorly reported and had no description of patient characteristics, with just a few comparing models' outputs with biopsies results. When lung biopsy is not available, lung-RADS could help standardise the comparisons between the human radiologist and the machine. The field of radiology should not give up principles from the diagnostic accuracy studies, such as the choice for the correct ground truth, just because AI is used. Clear and complete reporting of the reference standard used would help radiologists trust in the performance that AI models claim to have. This review presents clear recommendations about the essential methodological aspects of diagnostic models that should be incorporated in studies using AI to help detect or segmentate lung nodules. The manuscript also reinforces the need for more complete and transparent reporting, which can be helped using the recommended reporting guidelines.
Collapse
Affiliation(s)
| | | | | | | | | | - Garrett Bullock
- Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | | |
Collapse
|
9
|
Antoniou SA, Florez ID, Markar S, Logullo P, López-Cano M, Silecchia G, Antoniou GA, Tsokani S, Mavridis D, Brouwers M, Bertolaccini L, Alonso-Coello P, Akl E, Chand M, Como JJ, de Borst GJ, Di Saverio S, Emile S, Eom BW, Gorter R, Hanna G, Immonen K, Lai Q, Lumen N, Mathew JL, Montendori A, Moya M, Pellino G, Sanabria A, Saratzis A, Smart N, Stefanidis D, Zaninotto G. Author Correction: AGREE-S: AGREE II extension for surgical interventions: appraisal instrument. Surg Endosc 2023; 37:780. [PMID: 36414872 DOI: 10.1007/s00464-022-09770-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Affiliation(s)
- Stavros A Antoniou
- Department of Surgery, Mediterranean Hospital of Cyprus, Limassol, Cyprus. .,European University Cyprus, Nicosia, Cyprus.
| | - Ivan D Florez
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.,Department of Pediatrics, University of Antioquia, Medellin, Colombia
| | - Sheraz Markar
- Nuffield Department of Surgery, University of Oxford, Oxford, UK.,Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm, Sweden
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Manuel López-Cano
- Abdominal Wall Surgery Unit, Val d' Hebrón University Hospital, Universidad Autónoma de Barcelona, Barcelona, Spain
| | - Gianfranco Silecchia
- Department of Medico-Surgical Sciences and Translation Medicine, Faculty of Medicine and Psychology, Sapienza University of Rome, Rome, Italy
| | - George A Antoniou
- Department of Vascular and Endovascular Surgery, Manchester University NHS Foundation Trust, Manchester, UK.,Division of Cardiovascular Sciences, School of Medical Sciences, The University of Manchester, Manchester, UK
| | - Sofia Tsokani
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
| | - Dimitrios Mavridis
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece.,Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, Paris, France
| | - Melissa Brouwers
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
10
|
van Zuuren EJ, Logullo P, Price A, Fedorowicz Z, Hughes EL, Gattrell WT. Existing guidance on reporting of consensus methodology: a systematic review to inform ACCORD guideline development. BMJ Open 2022; 12:e065154. [PMID: 36201247 PMCID: PMC9462098 DOI: 10.1136/bmjopen-2022-065154] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 08/16/2022] [Indexed: 11/24/2022] Open
Abstract
OBJECTIVE To identify evidence on the reporting quality of consensus methodology and to select potential checklist items for the ACcurate COnsensus Reporting Document (ACCORD) project to develop a consensus reporting guideline. DESIGN Systematic review. DATA SOURCES Embase, MEDLINE, Web of Science, PubMed, Cochrane Library, Emcare, Academic Search Premier and PsycINFO from inception until 7 January 2022. ELIGIBILITY CRITERIA Studies, reviews and published guidance addressing the reporting quality of consensus methodology for improvement of health outcomes in biomedicine or clinical practice. Reports of studies using or describing consensus methods but not commenting on their reporting quality were excluded. No language restrictions were applied. DATA EXTRACTION AND SYNTHESIS Screening and data extraction of eligible studies were carried out independently by two authors. Reporting quality items addressed by the studies were synthesised narratively. RESULTS Eighteen studies were included: five systematic reviews, four narrative reviews, three research papers, three conference abstracts, two research guidance papers and one protocol. The majority of studies indicated that the quality of reporting of consensus methodology could be improved. Commonly addressed items were: consensus panel composition; definition of consensus and the threshold for achieving consensus. Items least addressed were: public patient involvement (PPI); the role of the steering committee, chair, cochair; conflict of interest of panellists and funding. Data extracted from included studies revealed additional items that were not captured in the data extraction form such as justification of deviation from the protocol or incentives to encourage panellist response. CONCLUSION The results of this systematic review confirmed the need for a reporting checklist for consensus methodology and provided a range of potential checklist items to report. The next step in the ACCORD project builds on this systematic review and focuses on reaching consensus on these items to develop the reporting guideline. PROTOCOL REGISTRATION https://osf.io/2rzm9.
Collapse
Affiliation(s)
- Esther J van Zuuren
- Department of Dermatology, Leiden University Medical Center, Leiden, Zuid-Holland, Netherlands
| | - Patricia Logullo
- Nuffield Department of Orthopaedics, Rheumatology and Muskuloskeletal Sciences, Centre for Statistics in Medicine, University of Oxford and EQUATOR Network UK Centre, Oxford, Oxfordshire, UK
| | - Amy Price
- Stanford Anesthesia, Informatics and Media Lab, Stanford University School of Medicine, Stanford, California, USA
- The BMJ, London, UK
| | | | | | | |
Collapse
|
11
|
Antoniou SA, Florez ID, Markar S, Logullo P, López-Cano M, Silecchia G, Antoniou GA, Tsokani S, Mavridis D, Brouwers M. AGREE-S: AGREE II extension for surgical interventions: appraisal instrument. Surg Endosc 2022; 36:5547-5558. [PMID: 35705753 DOI: 10.1007/s00464-022-09354-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 05/18/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND The Appraisal of Guidelines Research and Evaluation (AGREE) II instrument was developed to evaluate the quality of clinical practice guidelines. Evidence suggests that development, reporting, and appraisal of guidelines on surgical interventions may be better informed by modification of the instrument. OBJECTIVE We aimed to develop an AGREE II extension specifically designed for appraisal of guidelines of surgical interventions. METHODS In a three-part project funded by the United European Gastroenterology and the European Association for Endoscopic Surgery, (i) we identified factors that were associated with higher quality of surgical guidelines, (ii) we statistically calibrated the AGREE II instrument in the context of surgical guidelines using correlation, reliability, and factor analysis, and (iii) we undertook a Delphi consensus process of stakeholders to inform the development of an AGREE II extension instrument for surgical interventions. RESULTS Several features were prioritized by stakeholders as of particular importance for guidelines of surgical interventions, including development of a guideline protocol, consideration of practice variability and surgical expertise in different settings, and specification of infrastructures required to implement the recommendations. The AGREE-S-AGREE II extension instrument for surgical interventions has 25 items, compared to the 23 items of the original AGREE II instrument, organized into the following 6 domains: Scope and purpose, Stakeholders, Evidence synthesis, Development of recommendations, Editorial independence, and Implementation and update. As the original instrument, it concludes with an overall appraisal of the quality of the guideline and a judgement on whether the guideline is recommended for use. Several items were amended and rearranged among domains, and an item was deleted. The Rigor of Development domain of the original AGREE II was divided into Evidence Synthesis and Development of Recommendations. Items of the AGREE II domain Clarity of Presentation were incorporated in the new domain Development of Recommendations. Three new items were introduced, addressing the development of a guideline protocol, support by a guideline methodologist, and consideration of surgical experience/expertise. CONCLUSION The AGREE-S appraisal instrument has been developed to be used for assessment of the methodological and reporting quality of guidelines on surgical interventions.
Collapse
Affiliation(s)
- Stavros A Antoniou
- Department of Surgery, Mediterranean Hospital of Cyprus, Limassol, Cyprus.
- European University Cyprus, Nicosia, Cyprus.
| | - Ivan D Florez
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Department of Pediatrics, University of Antioquia, Medellin, Colombia
| | - Sheraz Markar
- Nuffield Department of Surgery, University of Oxford, Oxford, UK
- Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm, Sweden
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Manuel López-Cano
- Abdominal Wall Surgery Unit, Val d' Hebrón University Hospital, Universidad Autónoma de Barcelona, Barcelona, Spain
| | - Gianfranco Silecchia
- Department of Medico-Surgical Sciences and Translation Medicine, Faculty of Medicine and Psychology, Sapienza University of Rome, Rome, Italy
| | - George A Antoniou
- Department of Vascular and Endovascular Surgery, Manchester University NHS Foundation Trust, Manchester, UK
- Division of Cardiovascular Sciences, School of Medical Sciences, The University of Manchester, Manchester, UK
| | - Sofia Tsokani
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
| | - Dimitrios Mavridis
- Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
- Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, Paris, France
| | - Melissa Brouwers
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
12
|
Gattrell WT, Hungin AP, Price A, Winchester CC, Tovey D, Hughes EL, van Zuuren EJ, Goldman K, Logullo P, Matheis R, Harrison N. ACCORD guideline for reporting consensus-based methods in biomedical research and clinical practice: a study protocol. Res Integr Peer Rev 2022; 7:3. [PMID: 35672782 PMCID: PMC9171734 DOI: 10.1186/s41073-022-00122-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Accepted: 05/09/2022] [Indexed: 12/11/2022] Open
Abstract
BACKGROUND Structured, systematic methods to formulate consensus recommendations, such as the Delphi process or nominal group technique, among others, provide the opportunity to harness the knowledge of experts to support clinical decision making in areas of uncertainty. They are widely used in biomedical research, in particular where disease characteristics or resource limitations mean that high-quality evidence generation is difficult. However, poor reporting of methods used to reach a consensus - for example, not clearly explaining the definition of consensus, or not stating how consensus group panellists were selected - can potentially undermine confidence in this type of research and hinder reproducibility. Our objective is therefore to systematically develop a reporting guideline to help the biomedical research and clinical practice community describe the methods or techniques used to reach consensus in a complete, transparent, and consistent manner. METHODS The ACCORD (ACcurate COnsensus Reporting Document) project will take place in five stages and follow the EQUATOR Network guidance for the development of reporting guidelines. In Stage 1, a multidisciplinary Steering Committee has been established to lead and coordinate the guideline development process. In Stage 2, a systematic literature review will identify evidence on the quality of the reporting of consensus methodology, to obtain potential items for a reporting checklist. In Stage 3, Delphi methodology will be used to reach consensus regarding the checklist items, first among the Steering Committee, and then among a broader Delphi panel comprising participants with a range of expertise, including patient representatives. In Stage 4, the reporting guideline will be finalised in a consensus meeting, along with the production of an Explanation and Elaboration (E&E) document. In Stage 5, we plan to publish the reporting guideline and E&E document in open-access journals, supported by presentations at appropriate events. Dissemination of the reporting guideline, including a website linked to social media channels, is crucial for the document to be implemented in practice. DISCUSSION The ACCORD reporting guideline will provide a set of minimum items that should be reported about methods used to achieve consensus, including approaches ranging from simple unstructured opinion gatherings to highly structured processes.
Collapse
Affiliation(s)
| | | | - Amy Price
- Stanford Anesthesia, Informatics and Media Lab, Stanford University School of Medicine, Stanford, CA, USA
| | | | - David Tovey
- Journal of Clinical Epidemiology, Sussex, UK
| | | | | | - Keith Goldman
- Global Medical Affairs, AbbVie, North Chicago, IL, USA
| | - Patricia Logullo
- Centre for Statistics in Medicine (CSM), University of Oxford, and EQUATOR Network UK Centre, Oxford, UK
| | - Robert Matheis
- International Society for Medical Publication Professionals, New York, NY, USA
| | | |
Collapse
|
13
|
Bradley SH, DeVito NJ, Lloyd KE, Logullo P, Butler JE. Improving medical research in the United Kingdom. BMC Res Notes 2022; 15:165. [PMID: 35562775 PMCID: PMC9100293 DOI: 10.1186/s13104-022-06050-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 04/24/2022] [Indexed: 11/13/2022] Open
Abstract
Poor quality medical research causes serious harms by misleading healthcare professionals and policymakers, decreasing trust in science and medicine, and wasting public funds. Here we outline underlying problems including insufficient transparency, dysfunctional incentives, and reporting biases. We make the following recommendations to address these problems: Journals and funders should ensure authors fulfil their obligation to share detailed study protocols, analytical code, and (as far as possible) research data. Funders and journals should incentivise uptake of registered reports and establish funding pathways which integrate evaluation of funding proposals with initial peer review of registered reports. A mandatory national register of interests for all those who are involved in medical research in the UK should be established, with an expectation that individuals maintain the accuracy of their declarations and regularly update them. Funders and institutions should stop using metrics such as citations and journal's impact factor to assess research and researchers and instead evaluate based on quality, reproducibility, and societal value. Employers and non-academic training programmes for health professionals (clinicians hired for patient care, not to do research) should not select based on number of research publications. Promotions based on publication should be restricted to those hired to do research.
Collapse
Affiliation(s)
- Stephen H. Bradley
- Leeds Institute of Health Sciences, University of Leeds, Worsley Building, Leeds, LS2 9JT UK
| | - Nicholas J. DeVito
- The DataLab and Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, New Radcliffe House, 2nd floor, Radcliffe Observatory Quarter,Woodstock Road, Oxford, OX2 6GG UK
| | - Kelly E. Lloyd
- Leeds Institute of Health Sciences, University of Leeds, Worsley Building, Leeds, LS2 9JT UK
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Windmill Road, Oxford, OX3 7LD UK
| | - Jessica E. Butler
- Centre for Health Data Science, University of Aberdeen, Aberdeen, AB25 2ZD UK
| |
Collapse
|
14
|
Logullo P, Florez ID, Antoniou GA, Markar S, López‐Cano M, Silecchia G, Tsokani S, Mavridis D, Brouwers M, Antoniou SA. AGREE-S: AGREE II extension for surgical interventions - United European Gastroenterology and European Association for Endoscopic Surgery methodological guide. United European Gastroenterol J 2022; 10:425-434. [PMID: 35506366 PMCID: PMC9103371 DOI: 10.1002/ueg2.12231] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 03/31/2022] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND The Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument has been developed to inform the methodology, reporting and appraisal of clinical practice guidelines. Evidence suggests that the quality of surgical guidelines can be improved, and the structure and content of AGREE II can be modified to help enhance the quality of guidelines of surgical interventions. OBJECTIVE To develop an extension of AGREE II specifically designed for guidelines of surgical interventions. METHODS In the tripartite Guideline Assessment Project (GAP) funded by United European Gastroenterology and the European Association for Endoscopic Surgery, (i) we assessed the quality of surgical guidelines and we identified factors associated with higher quality (GAP I); (ii) we applied correlation analysis, factor analysis and the item response theory to inform an adaption of AGREE II for the purposes of surgical guidelines (GAP II); and (iii) we developed an AGREE II extension for surgical interventions, informed by the results of GAP I, GAP II, and a Delphi process of stakeholders, including representation from interventional and surgical disciplines; the Guideline International Network (GIN); the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) initiative; and representation of surgical journal editors and patient/public. RESULTS We developed AGREE-S, an AGREE II extension for surgical interventions, which comprises 24 items organized in 6 domains; Scope and purpose, Stakeholders, Evidence synthesis, Development of recommendations, Editorial independence, and Implementation and update. The panel of stakeholders proposed 3 additional items: development of a guideline protocol, consideration of practice variability and surgical/interventional expertise in different settings, and specification of infrastructures required to implement the recommendations. Three of the existing items were amended, 7 items were rearranged among the domains, and one item was removed. The domain Rigour of Development was divided into domains on Evidence Synthesis and Development of Recommendations. The new domain Development of Recommendations incorporates items from the original AGREE II domain Clarity of Presentation. CONCLUSION AGREE-S is an evidence-based and stakeholder-informed extension of the AGREE II instrument, that can be used as a guide for the development and adaption of guidelines on surgical interventions.
Collapse
Affiliation(s)
- Patricia Logullo
- Department Nuffield of Orthopaedics, Rheumatology & Musculoskeletal SciencesUK EQUATOR CentreCentre for Statistics in MedicineUniversity of OxfordOxfordUK
| | - Ivan D Florez
- Department of Health Research MethodsEvidence and ImpactMcMaster UniversityHamiltonOntarioCanada
- Department of PediatricsUniversity of AntioquiaMedellinColombia
| | - George A Antoniou
- Department of Vascular and Endovascular SurgeryManchester University NHS Foundation TrustManchesterUK
- Division of Cardiovascular SciencesSchool of Medical SciencesThe University of ManchesterManchesterUK
| | - Sheraz Markar
- Nuffield Department of SurgeryUniversity of OxfordOxfordOxfordshireUK
- Department of Molecular Medicine and SurgeryKarolinska InstituteStockholmSweden
| | - Manuel López‐Cano
- Abdominal Wall Surgery UnitVal d’ Hebrón University HospitalUniversidad Autónoma de BarcelonaBarcelonaSpain
| | - Gianfranco Silecchia
- Department of Medico‐Surgical Sciences and Translation MedicineFaculty of Medicine and PsychologySapienza University of RomeRomeItaly
| | - Sofia Tsokani
- Department of Primary EducationSchool of Education University of IoanninaIoanninaGreece
| | - Dimitrios Mavridis
- Department of Primary EducationSchool of Education University of IoanninaIoanninaGreece
- Paris Descartes UniversitySorbonne Paris CitéFaculté de MédecineParisFrance
| | - Melissa Brouwers
- Department of Health Research MethodsEvidence and ImpactMcMaster UniversityHamiltonOntarioCanada
| | - Stavros A Antoniou
- Department of SurgeryMediterranean Hospital of CyprusLimassolCyprus
- European University CyprusNicosiaCyprus
| |
Collapse
|
15
|
Struthers C, Harwood J, de Beyer JA, Dhiman P, Logullo P, Schlüssel M. GoodReports: developing a website to help health researchers find and use reporting guidelines. BMC Med Res Methodol 2021; 21:217. [PMID: 34657590 PMCID: PMC8520646 DOI: 10.1186/s12874-021-01402-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 09/13/2021] [Indexed: 12/18/2022] Open
Abstract
BACKGROUND Th EQUATOR Network improves the quality and transparency in health research, primarily by promoting awareness and use of reporting guidelines. In 2018, the UK EQUATOR Centre launched GoodReports.org , a website that helps authors find and use reporting guidelines. This paper describes the tool's development so far. We describe user experience and behaviour of using GoodReports.org both inside and outside a journal manuscript submission process. We intend to use our findings to inform future development and testing of the tool. METHODS We conducted a survey to collect data on user experience of the GoodReports website. We cross-checked a random sample of 100 manuscripts submitted to a partner journal to describe the level of agreement between the tool's checklist recommendation and what we would have recommended. We compared the proportion of authors submitting a completed reporting checklist alongside their manuscripts between groups exposed or not exposed to the GoodReports tool. We also conducted a study comparing completeness of reporting of manuscript text before an author received a reporting guideline recommendation from GoodReports.org with the completeness of the text subsequently submitted to a partner journal. RESULTS Seventy percent (423/599) of survey respondents rated GoodReports 8 or more out of 10 for usefulness, and 74% (198/267) said they had made changes to their manuscript after using the website. We agreed with the GoodReports reporting guideline recommendation in 84% (72/86) of cases. Of authors who completed the guideline finder questionnaire, 14% (10/69) failed to submit a completed checklist compared to 30% (41/136) who did not use the tool. Of the 69 authors who received a GoodReports reporting guideline recommendation, 20 manuscript pairs could be reviewed before and after use of GoodReports. Five included more information in their methods section after exposure to GoodReports. On average, authors reported 57% of necessary reporting items before completing a checklist on GoodReports.org and 60% after. CONCLUSION The data suggest that reporting guidance is needed early in the writing process, not at submission stage. We are developing GoodReports by adding more reporting guidelines and by creating editable article templates. We will test whether GoodReports users write more complete study reports in a randomised trial targeting researchers starting to write health research articles.
Collapse
Affiliation(s)
- Caroline Struthers
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK.
| | - James Harwood
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK
| | - Jennifer Anne de Beyer
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK
| | - Paula Dhiman
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK
| | - Michael Schlüssel
- UK EQUATOR Centre, Centre for Statistics in Medicine, NDORMS, University of Oxford, Oxford, UK
| |
Collapse
|
16
|
Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, Logullo P, Beam AL, Peng L, Van Calster B, van Smeden M, Riley RD, Moons KG. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open 2021; 11:e048008. [PMID: 34244270 PMCID: PMC8273461 DOI: 10.1136/bmjopen-2020-048008] [Citation(s) in RCA: 236] [Impact Index Per Article: 78.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 06/23/2021] [Indexed: 12/18/2022] Open
Abstract
INTRODUCTION The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were both published to improve the reporting and critical appraisal of prediction model studies for diagnosis and prognosis. This paper describes the processes and methods that will be used to develop an extension to the TRIPOD statement (TRIPOD-artificial intelligence, AI) and the PROBAST (PROBAST-AI) tool for prediction model studies that applied machine learning techniques. METHODS AND ANALYSIS TRIPOD-AI and PROBAST-AI will be developed following published guidance from the EQUATOR Network, and will comprise five stages. Stage 1 will comprise two systematic reviews (across all medical fields and specifically in oncology) to examine the quality of reporting in published machine-learning-based prediction model studies. In stage 2, we will consult a diverse group of key stakeholders using a Delphi process to identify items to be considered for inclusion in TRIPOD-AI and PROBAST-AI. Stage 3 will be virtual consensus meetings to consolidate and prioritise key items to be included in TRIPOD-AI and PROBAST-AI. Stage 4 will involve developing the TRIPOD-AI checklist and the PROBAST-AI tool, and writing the accompanying explanation and elaboration papers. In the final stage, stage 5, we will disseminate TRIPOD-AI and PROBAST-AI via journals, conferences, blogs, websites (including TRIPOD, PROBAST and EQUATOR Network) and social media. TRIPOD-AI will provide researchers working on prediction model studies based on machine learning with a reporting guideline that can help them report key details that readers need to evaluate the study quality and interpret its findings, potentially reducing research waste. We anticipate PROBAST-AI will help researchers, clinicians, systematic reviewers and policymakers critically appraise the design, conduct and analysis of machine learning based prediction model studies, with a robust standardised tool for bias evaluation. ETHICS AND DISSEMINATION Ethical approval has been granted by the Central University Research Ethics Committee, University of Oxford on 10-December-2020 (R73034/RE001). Findings from this study will be disseminated through peer-review publications. PROSPERO REGISTRATION NUMBER CRD42019140361 and CRD42019161764.
Collapse
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, NIHR Oxford Biomedical Research Centre, Oxford, UK
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, NIHR Oxford Biomedical Research Centre, Oxford, UK
| | | | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
| | - Patricia Logullo
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, NIHR Oxford Biomedical Research Centre, Oxford, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| | - Lily Peng
- Google Health, Google, Palo Alto, California, USA
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands
- EPI-Centre, KU Leuven, Leuven, Belgium
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Karel Gm Moons
- Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, Utrecht, Utrecht, Netherlands
| |
Collapse
|
17
|
MacCarthy A, Dhiman P, Kirtley S, Logullo P, Copsey B, Collins GS. A review found inadequate reporting of case-control studies of risk factors for pancreatic cancer. J Clin Epidemiol 2021; 133:32-42. [PMID: 33359318 PMCID: PMC8168827 DOI: 10.1016/j.jclinepi.2020.12.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Revised: 12/08/2020] [Accepted: 12/15/2020] [Indexed: 11/30/2022]
Abstract
OBJECTIVES Case-control studies are often used to identify the risk factors for pancreatic cancer. The objective of this study was to evaluate the reporting of case-control studies of the risk factors for pancreatic cancer using the Strengthening The Reporting of OBservational Studies in Epidemiology (STROBE) for case-control studies checklist. STUDY DESIGN AND SETTING We conducted a comprehensive literature search of the MEDLINE and EMBASE databases to identify reports of case-control studies published between 2016 and 2018. We scored article reporting using a reporting adherence form developed from the STROBE checklist for case-control studies, consisting of 14 STROBE items related to the title, abstract, methods, and results sections. RESULTS We included reports of 47 case-control studies investigating a variety of risk factors, such as medical conditions and lifestyle factors. Reporting was inconsistent and inadequate. Efforts to address bias and how the study size was arrived at were particularly poorly described. Study cases were described in more detail than study controls. CONCLUSION Reporting of case-control studies remains inadequate more than 10 years after the STROBE reporting guideline was published. Our findings suggest that authors do not understand the extent to which study methods and findings should be reported to enable studies to be fully understood, and their methods reproduced.
Collapse
Affiliation(s)
- Angela MacCarthy
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK.
| | - Paula Dhiman
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Shona Kirtley
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK
| | - Patricia Logullo
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Bethan Copsey
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford OX3 9DU, UK
| |
Collapse
|
18
|
Speich B, Logullo P, Deuster S, Marian IR, Moschandreas J, Taji Heravi A, Gloy V, Briel M, Hopewell S. A meta-research study revealed several challenges in obtaining placebos for investigator-initiated drug trials. J Clin Epidemiol 2021; 131:70-78. [PMID: 33242608 DOI: 10.1016/j.jclinepi.2020.11.007] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 10/16/2020] [Accepted: 11/13/2020] [Indexed: 12/26/2022]
Abstract
OBJECTIVES To systematically assess the kind of placebos used in investigator-initiated randomized controlled trials (RCTs), from where they are obtained, and the hurdles that exist in obtaining them. STUDY DESIGN AND SETTING PubMed was searched for recently published noncommercial, placebo-controlled randomized drug trials. Corresponding authors were invited to participate in an online survey. RESULTS From 423 eligible articles, 109 (26%) corresponding authors (partially) participated. Twenty-one of 102 (21%) authors reported that the placebos used were not matching (correctly labeled in only one publication). The main sources in obtaining placebos were hospital pharmacies (32 of 107; 30%) and the manufacturer of the study drug (28 of 107; 26%). RCTs with a hypothesis in the interest of the manufacturer of the study drug were more likely to have obtained placebos from the drug manufacturer (18 of 49; 37% vs. 5 of 29; 17%). Median costs for placebos and packaging were US$ 58,286 (IQR US$ 2,428- US$ 160,770; n = 24), accounting for a median of 10.3% of the overall trial budget. CONCLUSION Although using matching placebos is widely accepted as a basic practice in RCTs, there seems to be no standard source to acquire them. Obtaining placebos requires substantial resources, and using nonmatching placebos is common.
Collapse
Affiliation(s)
- Benjamin Speich
- Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom; Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland.
| | - Patricia Logullo
- Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom; The EQUATOR Network, Oxford, United Kingdom
| | - Stefanie Deuster
- Hospital Pharmacy, University Hospital Basel, Basel, Switzerland
| | - Ioana R Marian
- Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Joanna Moschandreas
- Nuffield Department of Clinical Neurosciences, Centre for the Prevention of Stroke and Dementia, University of Oxford, Oxford, United Kingdom
| | - Ala Taji Heravi
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland; Swiss Tropical and Public Health Institute, Basel, Switzerland
| | - Viktoria Gloy
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Matthias Briel
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
| | - Sally Hopewell
- Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
19
|
Affiliation(s)
- Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Jennifer A de Beyer
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Caroline Struthers
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Gary S Collins
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
- National Institute for Health Research Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, United Kingdom
| |
Collapse
|
20
|
Logullo P, MacCarthy A, Kirtley S, Collins GS. Reporting guideline checklists are not quality evaluation forms: they are guidance for writing. Health Sci Rep 2020; 3:e165. [PMID: 32373717 PMCID: PMC7196677 DOI: 10.1002/hsr2.165] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2020] [Accepted: 04/09/2020] [Indexed: 12/19/2022] Open
Affiliation(s)
- Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics & Musculoskeletal SciencesUniversity of OxfordOxfordUK
| | - Angela MacCarthy
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics & Musculoskeletal SciencesUniversity of OxfordOxfordUK
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics & Musculoskeletal SciencesUniversity of OxfordOxfordUK
| | - Gary S. Collins
- UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics & Musculoskeletal SciencesUniversity of OxfordOxfordUK
- NIHR Oxford Biomedical Research CentreJohn Radcliffe HospitalOxfordUK
| |
Collapse
|
21
|
Mozetic V, Leonel L, Leite Pacheco R, de Oliveira Cruz Latorraca C, Guimarães T, Logullo P, Riera R. Reporting quality and adherence of randomized controlled trials about statins and/or fibrates for diabetic retinopathy to the CONSORT checklist. Trials 2019; 20:729. [PMID: 31842982 PMCID: PMC6916100 DOI: 10.1186/s13063-019-3868-4] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2019] [Accepted: 10/31/2019] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND A considerable amount of randomized controlled trials (RCTs) have been published on statins and/or fibrates for diabetic retinopathy, a clinical condition associated with high social and economic burden. Adherence to the CONSORT statement items is imperative to ensure transparency and reproducibility in clinical research. The aim of this study is to assess the reporting quality and the adherence to CONSORT of RCTs assessing statins and/or fibrates for diabetic retinopathy. METHODS We conducted a critical appraisal study at Discipline of Evidence-based Medicine, Escola Paulista de Medicina, Universidade Federal de São Paulo (Unifesp). A sensitive literature search was performed to identify all relevant RCTs, with no time or language limits. Two authors independently evaluated the reporting quality of the selected RCTs using the CONSORT statement as a standard. RESULTS Thirteen reports of RCTs were included in this study. The adherence of the reports to CONSORT items ranged from 24% to 68%. The median score was 11 (interquartile range (IQR) 8 to 13). When analyzed separately, the methods sections of the reports had a median of three items (IQR 2 to 4) judged adherent to the methods items of CONSORT (items 3 to 12). The most underreported items were those related to trial design, title and abstract, allocation concealment, implementation of the randomization sequence, and blinding. Other important items, such as the one related to the description of the inclusion criteria, also had low adherence. CONCLUSIONS The overall adherence to the CONSORT checklist items was poor, especially in the items related to the methods section. RCT reports on statins and/or fibrates for diabetic retinopathy must be optimized to avoid reporting biases and to improve transparency and reproducibility.
Collapse
Affiliation(s)
- Vânia Mozetic
- Ophthalmologist of Instituto Dante Pazzanese de Cardiologia, Sao Paulo, Brazil
| | - Letícia Leonel
- Discipline of Evidence-Based Medicine, Universidade Federal de São Paulo (Unifesp), São Paulo, Brasil
| | - Rafael Leite Pacheco
- Discipline of Evidence-Based Medicine, Universidade Federal de São Paulo (Unifesp), São Paulo, Brasil
- Centro Universitário São Camilo, São Paulo, Brazil
| | | | - Taís Guimarães
- Discipline of Evidence-Based Medicine, Universidade Federal de São Paulo (Unifesp), São Paulo, Brasil
| | | | - Rachel Riera
- Discipline of Evidence-Based Medicine, Universidade Federal de São Paulo (Unifesp), São Paulo, Brasil
- Centre of Health Technology Assessment, Hospital Sírio-Libanês, São Paulo, Brazil
| |
Collapse
|
22
|
Logullo P, Torloni MR, de O C Latorraca C, Riera R. The Brazilian Portuguese Version of the DISCERN Instrument: Translation Procedures and Psychometric Properties. Value Health Reg Issues 2019; 20:172-179. [PMID: 31622803 DOI: 10.1016/j.vhri.2019.09.001] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Accepted: 09/02/2019] [Indexed: 11/24/2022]
Abstract
OBJECTIVES To report on the translation procedures and psychometric properties of the DISCERN tool in Brazilian Portuguese. METHODS Three people translated the DISCERN from English into Brazilian Portuguese. A committee of experts and community representatives evaluated the quality of the 3 versions in 2 online voting rounds. Two native speakers back-translated the questionnaire into English. We compared these versions to the original DISCERN and made small adjustments. The final Brazilian Portuguese version of DISCERN was tested twice by journalism students to evaluate the quality of a text about smoking cessation treatments. We evaluated participants' health literacy with the Short Assessment of Health Literacy for Portuguese-Speaking Adults (SAHL-PA) tool, assessed the internal consistency of the translated questionnaire with the Cronbach test, and measured its reproducibility with the intraclass correlation coefficient (ICC). We then investigated the relationship between DISCERN and SAHL-PA scores and demographic variables. RESULTS The participants (n = 126) had no difficulty in using the questionnaire. Cronbach's alpha was 0.865 (95% confidence interval [CI], 0.826-0.898), and the ICC between the 2 evaluations was 0.845 (CI 0.717-0.912). The mean health literacy of the participants was adequate. There was no correlation between the DISCERN score and the SAHL-PA score, age, or sex (P > .05). CONCLUSIONS The Brazilian Portuguese version of the DISCERN questionnaire has excellent internal consistency and good reproducibility. The evaluators' ages, sex, and health literacy did not interfere with the score resulting from the evaluation of the quality of the text.
Collapse
Affiliation(s)
- Patricia Logullo
- Discipline of Evidence-Based Medicine, Escola Paulista de Medicina, and Post-Graduation Program of Evidence-Based Health, Universidade Federal de São Paulo, São Paulo, Brazil; Cochrane Brazil, São Paulo, Brazil; EQUATOR Network, University of Oxford, Oxford, England, UK.
| | - Maria Regina Torloni
- Discipline of Evidence-Based Medicine, Escola Paulista de Medicina, and Post-Graduation Program of Evidence-Based Health, Universidade Federal de São Paulo, São Paulo, Brazil; Cochrane Brazil, São Paulo, Brazil
| | - Carolina de O C Latorraca
- Discipline of Evidence-Based Medicine, Escola Paulista de Medicina, and Post-Graduation Program of Evidence-Based Health, Universidade Federal de São Paulo, São Paulo, Brazil; Cochrane Brazil, São Paulo, Brazil
| | - Rachel Riera
- Discipline of Evidence-Based Medicine, Escola Paulista de Medicina, and Post-Graduation Program of Evidence-Based Health, Universidade Federal de São Paulo, São Paulo, Brazil; Cochrane Brazil, São Paulo, Brazil; Center of Health Technological Assessment, Instituto Sírio-Libanês de Ensino e Pesquisa, Hospital Sírio-Libanês, São Paulo, Brazil
| |
Collapse
|
23
|
Quintão VC, Logullo P, Schlüssel MM, Kirtley S, Collins G, Carmona MJC. Reporting guidelines: tools to increase the completeness and transparency of your anesthesiology research paper. Brazilian Journal of Anesthesiology (English Edition) 2019. [PMID: 31630849 PMCID: PMC9391902 DOI: 10.1016/j.bjane.2019.09.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
24
|
Quintão VC, Logullo P, Schlüssel MM, Kirtley S, Collins G, Carmona MJC. [Reporting guidelines: tools to increase the completeness and transparency of your anesthesiology research paper]. Braz J Anesthesiol 2019; 69:429-431. [PMID: 31630849 PMCID: PMC9391902 DOI: 10.1016/j.bjan.2019.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2019] [Indexed: 10/25/2022]
Affiliation(s)
- Vinícius Caldeira Quintão
- Universidade de São Paulo, Faculdade de Medicina, Disciplina de Anestesiologia, São Paulo, SP, Brasil
| | - Patricia Logullo
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Oxford, Reino Unido
| | - Michael Maia Schlüssel
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Oxford, Reino Unido
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Oxford, Reino Unido
| | - Gary Collins
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Oxford, Reino Unido
| | | |
Collapse
|
25
|
Logullo P, Barbosa de Carvalho H, Saconi R, Massad E. Factors affecting compliance with the measles vaccination schedule in a Brazilian city. SAO PAULO MED J 2008; 126:166-71. [PMID: 18711656 DOI: 10.1590/s1516-31802008000300006] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/28/2007] [Accepted: 03/31/2008] [Indexed: 11/22/2022] Open
Abstract
CONTEXT AND OBJECTIVE The success of vaccination campaigns depends on the degree of adherence to immunization initiatives and schedules. Risk factors associated with children's failure to receive the measles vaccine at the correct age were studied in the city of São Paulo, Brazil. DESIGN AND SETTING Case-control and exploratory study, in the metropolitan area of São Paulo. METHODS The caregivers of 122 children were interviewed regarding their perceptions and understanding about the measles vaccination and the disease. RESULTS The results showed that age, region of residence, marital status and education level were unrelated to taking measles vaccines adequately. Most individuals remembered being informed about the last annual vaccination campaign by television, but no communication channel was significantly associated with vaccination status. The answers to questions about knowledge of the disease or the vaccine, when analyzed alone, were not associated with taking measles vaccinations at the time indicated by health agencies. The results showed that, when parents felt sorry for their children who were going to receive shots, they delayed the vaccination. Most of the children did not take the measles vaccination on the exactly recommended date, but delayed or anticipated the shots. CONCLUSION It is clear that there is no compliance with the government's recommended measles vaccination schedule (i.e. first dose at nine and second at 15 months of age, as recommended in 1999 and 2000). Feeling sorry for the children receiving shots can delay vaccination taking.
Collapse
Affiliation(s)
- Patricia Logullo
- Department of Medical Computer Science, Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brazil.
| | | | | | | |
Collapse
|
26
|
Logullo P, de Carvalho HB, Saconi R, Massad E. Fear of injections is a reason for not to vaccinate their children, say caretakers. Vaccine 2008; 26:141-3. [DOI: 10.1016/j.vaccine.2007.09.075] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2007] [Accepted: 09/03/2007] [Indexed: 10/22/2022]
|