1
|
Stopsack KH, Mucci LA, Tworoger SS, Kang JH, Eliassen AH, Willett WC, Stampfer MJ. Promoting Reproducibility and Integrity in Observational Research: One Approach of an Epidemiology Research Community. Epidemiology 2023; 34:389-395. [PMID: 36719725 PMCID: PMC10073307 DOI: 10.1097/ede.0000000000001599] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2022] [Accepted: 01/27/2023] [Indexed: 02/01/2023]
Abstract
To increase research reproducibility, sharing of study data, analysis code, and use of standardized reporting are increasingly advocated. However, beyond reproducibility, few initiatives have addressed the integrity of how research is conducted before manuscripts are submitted. We describe a decades-long experience with a comprehensive approach based in an academic research community around prospective cohort studies that is aimed at promoting a culture of integrity in observational research. The approach includes prespecifying hypotheses and analysis plans, which are discussed in the research community and posted; presentation and discussion of analysis results; mandatory analysis code review by a programmer; review of concordance between analysis output and manuscripts by a technical reviewer; and checks of adherence to the process, including compliance with institutional review board requirements and reporting stipulations by the National Institutes of Health. The technical core is based in shared computing and analytic environments with long-term archiving. More than simply a list of rules, our approach promotes research integrity through integrated educational elements, making it part of the "hidden curriculum," by fostering a sense of belonging, and by providing efficiency gains to the research community. Unlike reproducibility checklists, such long-term investments into research integrity require substantial and sustained funding for research personnel and computing infrastructure. Our experiences suggest avenues for how institutions, research communities, and funders involved in observational research can strengthen integrity within the research process.
Collapse
Affiliation(s)
- Konrad H Stopsack
- From the Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
- Clinical and Translational Epidemiology Unit, Massachusetts General Hospital and Harvard Medical School, Boston, MA
| | - Lorelei A Mucci
- From the Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
| | | | - Jae H Kang
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA
| | - A Heather Eliassen
- From the Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA
- Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA
| | - Walter C Willett
- From the Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
- Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA
| | - Meir J Stampfer
- From the Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA
- Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA
| |
Collapse
|
2
|
Affiliation(s)
- Timothy L Lash
- From the Department of Epidemiology, Rollins School of Public Health, Atlanta, GA
| |
Collapse
|
3
|
Ramke J, Kuper H, Limburg H, Kinloch J, Zhu W, Lansingh VC, Congdon N, Foster A, Gilbert CE. Avoidable Waste in Ophthalmic Epidemiology: A Review of Blindness Prevalence Surveys in Low and Middle Income Countries 2000-2014. Ophthalmic Epidemiol 2017; 25:13-20. [PMID: 28886260 DOI: 10.1080/09286586.2017.1328067] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
PURPOSE Sources of avoidable waste in ophthalmic epidemiology include duplication of effort, and survey reports remaining unpublished, gaining publication after a long delay, or being incomplete or of poor quality. The aim of this review was to assess these sources of avoidable waste by examining blindness prevalence surveys undertaken in low and middle income countries (LMICs) between 2000 and 2014. METHODS On December 1, 2016 we searched MEDLINE, EMBASE and Web of Science databases for cross-sectional blindness prevalence surveys undertaken in LMICs between 2000 and 2014. All surveys listed on the Rapid Assessment of Avoidable Blindness (RAAB) Repository website ("the Repository") were also considered. For each survey we assessed (1) availability of scientific publication, survey report, summary results tables and/or datasets; (2) time to publication from year of survey completion and journal attributes; (3) extent of blindness information reported; and (4) rigour when information was available from two sources (i.e. whether it matched). RESULTS Of the 279 included surveys (from 68 countries) 186 (67%) used RAAB methodology; 146 (52%) were published in a scientific journal, 57 (20%) were published in a journal and on the Repository, and 76 (27%) were on the Repository only (8% had tables; 19% had no information available beyond registration). Datasets were available for 50 RAABs (18% of included surveys). Time to publication ranged from <1 to 11 years (mean, standard deviation 2.8 ± 1.8 years). The extent of blindness information reported within studies varied (e.g. presenting and best-corrected, unilateral and bilateral); those with both a published report and Repository tables were most complete. For surveys published and with RAAB tables available, discrepancies were found in reporting of participant numbers (14% of studies) and blindness prevalence (15%). CONCLUSION Strategies are needed to improve the availability, consistency, and quality of information reported from blindness prevalence surveys, and hence reduce avoidable waste.
Collapse
Affiliation(s)
- Jacqueline Ramke
- a University of Auckland, School of Population Health , Faculty of Medicine and Health Sciences , Auckland , New Zealand
| | - Hannah Kuper
- b London School of Hygiene and Tropical Medicine, International Centre for Eye Health, Clinical Research Unit , Department of Infectious & Tropical Diseases , London , UK
| | - Hans Limburg
- c Health Information Services , Grootebroek , Netherlands
| | | | - Wenhui Zhu
- e The First Affiliated Hospital of Sun Yat-sen University , Department of Ophthalmology , Guangzhou , Guangdong , China
| | - Van C Lansingh
- f Help Me See Latin America , Instituto Mexicano de Oftalmologia , Queretaro , Mexico
| | - Nathan Congdon
- g Queen's University Belfast, TREE , Centre for Public Health , Belfast , UK.,h Sun Yat-sen University, Preventive Ophthalmology Department , Zhongshan Ophthalmic Center , Guangzhou , China.,i Orbis International , NY , USA
| | - Allen Foster
- b London School of Hygiene and Tropical Medicine, International Centre for Eye Health, Clinical Research Unit , Department of Infectious & Tropical Diseases , London , UK
| | - Clare E Gilbert
- b London School of Hygiene and Tropical Medicine, International Centre for Eye Health, Clinical Research Unit , Department of Infectious & Tropical Diseases , London , UK
| |
Collapse
|
4
|
Hörnell A, Berg C, Forsum E, Larsson C, Sonestedt E, Åkesson A, Lachat C, Hawwash D, Kolsteren P, Byrnes G, De Keyzer W, Van Camp J, Cade JE, Greenwood DC, Slimani N, Cevallos M, Egger M, Huybrechts I, Wirfält E. Perspective: An Extension of the STROBE Statement for Observational Studies in Nutritional Epidemiology (STROBE-nut): Explanation and Elaboration. Adv Nutr 2017; 8:652-678. [PMID: 28916567 PMCID: PMC5593101 DOI: 10.3945/an.117.015941] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2017] [Revised: 05/15/2017] [Accepted: 07/13/2017] [Indexed: 01/28/2023] Open
Abstract
Nutritional epidemiology is an inherently complex and multifaceted research area. Dietary intake is a complex exposure and is challenging to describe and assess, and links between diet, health, and disease are difficult to ascertain. Consequently, adequate reporting is necessary to facilitate comprehension, interpretation, and generalizability of results and conclusions. The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement is an international and collaborative initiative aiming to enhance the quality of reporting of observational studies. We previously presented a checklist of 24 reporting recommendations for the field of nutritional epidemiology, called "the STROBE-nut." The STROBE-nut is an extension of the general STROBE statement, intended to complement the STROBE recommendations to improve and standardize the reporting in nutritional epidemiology. The aim of the present article is to explain the rationale for, and elaborate on, the STROBE-nut recommendations to enhance the clarity and to facilitate the understanding of the guidelines. Examples from the published literature are used as illustrations, and references are provided for further reading.
Collapse
Affiliation(s)
- Agneta Hörnell
- Department of Food and Nutrition, Umeå University, Umeå, Sweden
| | - Christina Berg
- Department of Food and Nutrition, and Sport Science, University of Gothenburg, Gothenburg, Sweden
| | - Elisabet Forsum
- Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
| | - Christel Larsson
- Department of Food and Nutrition, and Sport Science, University of Gothenburg, Gothenburg, Sweden
| | - Emily Sonestedt
- Department of Clinical Sciences in Malmö, Lund University, Malmö, Sweden
| | - Agneta Åkesson
- Institute of Environmental Medicine, Karolinska Institutet, Stockholm, Sweden
| | - Carl Lachat
- Department of Food Safety and Food Quality, Ghent University, Ghent, Belgium
| | - Dana Hawwash
- Department of Food Safety and Food Quality, Ghent University, Ghent, Belgium
| | - Patrick Kolsteren
- Department of Food Safety and Food Quality, Ghent University, Ghent, Belgium
| | - Graham Byrnes
- International Agency for Research on Cancer, Lyon, France
| | - Willem De Keyzer
- Department of Biosciences and Food Sciences, University College Ghent, Ghent, Belgium
| | - John Van Camp
- Department of Food Safety and Food Quality, Ghent University, Ghent, Belgium
| | - Janet E Cade
- Nutritional Epidemiology Group, School of Food Science and Nutrition, and
| | - Darren C Greenwood
- Biostatistics Unit, School of Medicine, University of Leeds, Leeds, United Kingdom
| | - Nadia Slimani
- International Agency for Research on Cancer, Lyon, France
| | - Myriam Cevallos
- Department of Clinical Research, University of Bern, Bern, Switzerland
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Matthias Egger
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | | | - Elisabet Wirfält
- Department of Clinical Sciences in Malmö, Lund University, Malmö, Sweden
| |
Collapse
|
5
|
LaKind JS, Goodman M, Makris SL, Mattison DR. Improving Concordance in Environmental Epidemiology: A Three-Part Proposal. JOURNAL OF TOXICOLOGY AND ENVIRONMENTAL HEALTH. PART B, CRITICAL REVIEWS 2015; 18:105-20. [PMID: 26158301 PMCID: PMC4733943 DOI: 10.1080/10937404.2015.1051612] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
In observational research, evidence is usually derived from multiple studies, and any single result is rarely considered sufficient for public health decision making. Despite more than five decades of research and thousands of studies published, the ability to draw robust conclusions regarding the presence or absence of causal links between specific environmental exposures and human health remains limited. To develop policies that are protective of public health and can withstand scrutiny, agencies need to rely on investigations of satisfactory quality that follow sufficiently concordant protocols in terms of exposure assessment, outcome ascertainment, data analysis, and reporting of results. Absent such concordance, the ability of environmental epidemiology studies to inform decision making is greatly diminished. Systems and tools are proposed here to improve concordance among environmental epidemiology studies. Specifically, working systems in place in other fields of research are critically examined and used as guidelines to develop analogous policies and procedures for environmental epidemiology. A three-part path forward toward more concordant, transparent, and readily accessible environmental epidemiology evidence that parallels ongoing efforts in medical research is proposed. The three parts address methods for improving quality and accessibility of systematic reviews, access to information on ongoing and completed studies, and principles for reporting. The goals are to increase the value of epidemiological research in public health decision making and to stimulate discussions around solutions proposed herein.
Collapse
Affiliation(s)
- Judy S. LaKind
- LaKind Associates, LLC, Catonsville, Maryland, USA
- Department of Epidemiology and Public Health, University of Maryland School of Medicine, Baltimore, Maryland, USA
- Department of Pediatrics, Penn State University College of Medicine, Milton S. Hershey Medical Center, Hershey, Pennsylvania, USA
| | - Michael Goodman
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, USA
| | - Susan L. Makris
- U.S. Environmental Protection Agency, National Center for Environmental Assessment, Washington, DC, USA
| | - Donald R. Mattison
- Risk Sciences International, Ottawa, Ontario, Canada
- McLaughlin Centre for Population Health Risk Assessment, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
6
|
Dijkers MP, Murphy SL, Krellman J. Evidence-based practice for rehabilitation professionals: concepts and controversies. Arch Phys Med Rehabil 2012; 93:S164-76. [PMID: 22683207 DOI: 10.1016/j.apmr.2011.12.014] [Citation(s) in RCA: 61] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2011] [Revised: 12/20/2011] [Accepted: 12/21/2011] [Indexed: 10/28/2022]
Abstract
This article describes evidence-based practice (EBP) in the health professions and sciences in general and in the rehabilitation disciplines specifically. It discusses the following: what counts as evidence and how that has changed over the last 4 decades, trends in the short history of evidence-based medicine and EBP, the fallacious nature of most criticisms of EBP, (perceived) shortcomings of clinical research and the resulting evidence in rehabilitation, resources available to clinicians who want their practice to be evidence-based, and the barriers these clinicians face in keeping up with the evidence and applying it in practice. Lastly, it describes how the development of a new art and science, knowledge translation, may play a role in truly making EBP feasible in rehabilitation services.
Collapse
Affiliation(s)
- Marcel P Dijkers
- Department of Rehabilitation Medicine, Mount Sinai School of Medicine, New York, NY 10029-6574, USA.
| | | | | |
Collapse
|
7
|
|
8
|
Savitz DA. Registration of Observational Studies Does Not Enhance Validity. Clin Pharmacol Ther 2011; 90:646-8. [DOI: 10.1038/clpt.2011.199] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
9
|
Ellenberg SS, Lewis JD. Registration of epidemiological studies: benefits and risks. Pharmacoepidemiol Drug Saf 2011; 20:1005-8. [DOI: 10.1002/pds.2111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2010] [Revised: 12/30/2010] [Accepted: 01/09/2011] [Indexed: 11/11/2022]
|
10
|
Chavers S, Fife D, Wacholtz M, Stang P, Berlin J. Registration of Observational Studies: perspectives from an industry-based epidemiology group. Pharmacoepidemiol Drug Saf 2011; 20:1009-13. [PMID: 21953845 DOI: 10.1002/pds.2221] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2010] [Revised: 06/22/2011] [Accepted: 06/27/2011] [Indexed: 11/10/2022]
Abstract
Registration of randomized clinical trials has become standard practice and is enforced through publication policies and governmental regulations. However, the registration of observational studies remains controversial. In this commentary, we propose that a compromise can be reached on which observation should be registered based on study design and study intent.
Collapse
Affiliation(s)
- Scott Chavers
- Johnson & Johnson Pharmaceutical Research and Development, Titusville, NJ, USA.
| | | | | | | | | |
Collapse
|
11
|
Making observational studies count: shaping the future of comparative effectiveness research. Epidemiology 2011; 22:295-7. [PMID: 21464648 DOI: 10.1097/ede.0b013e3182126569] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
12
|
|
13
|
|