1
|
Veronesi G, Grassi G, Savelli G, Quatto P, Zambon A. Big data, observational research and P-value: a recipe for false-positive findings? A study of simulated and real prospective cohorts. Int J Epidemiol 2021; 49:876-884. [PMID: 31620789 PMCID: PMC7394945 DOI: 10.1093/ije/dyz206] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/11/2019] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND An increasing number of observational studies combine large sample sizes with low participation rates, which could lead to standard inference failing to control the false-discovery rate. We investigated if the 'empirical calibration of P-value' method (EPCV), reliant on negative controls, can preserve type I error in the context of survival analysis. METHODS We used simulated cohort studies with 50% participation rate and two different selection bias mechanisms, and a real-life application on predictors of cancer mortality using data from four population-based cohorts in Northern Italy (n = 6976 men and women aged 25-74 years at baseline and 17 years of median follow-up). RESULTS Type I error for the standard Cox model was above the 5% nominal level in 15 out of 16 simulated settings; for n = 10 000, the chances of a null association with hazard ratio = 1.05 having a P-value < 0.05 were 42.5%. Conversely, EPCV with 10 negative controls preserved the 5% nominal level in all the simulation settings, reducing bias in the point estimate by 80-90% when its main assumption was verified. In the real case, 15 out of 21 (71%) blood markers with no association with cancer mortality according to literature had a P-value < 0.05 in age- and gender-adjusted Cox models. After calibration, only 1 (4.8%) remained statistically significant. CONCLUSIONS In the analyses of large observational studies prone to selection bias, the use of empirical distribution to calibrate P-values can substantially reduce the number of trivial results needing further screening for relevance and external validity.
Collapse
Affiliation(s)
- Giovanni Veronesi
- Research Center in Epidemiology and Preventive Medicine, Department of Medicine and Surgery, University of Insubria, Varese, Italy
| | - Guido Grassi
- Clinica Medica, Department of Medicine and Surgery, University of Milano-Bicocca, Milano, Italy
| | - Giordano Savelli
- U.O. Medicina Nucleare, Fondazione Poliambulanza Istituto Ospedaliero, Brescia, Italy
| | - Piero Quatto
- Department of Economics, Management and Statistics
| | - Antonella Zambon
- Department of Statistics and Quantitative Methods, University of Milano-Bicocca, Milano, Italy
| |
Collapse
|
2
|
Park CH, Winglee M, Kwan J, Andrews L, Hudak ML. Comparison of Recruitment Strategy Outcomes in the National Children's Study. Pediatrics 2017; 140:peds.2016-2822. [PMID: 28724571 PMCID: PMC5527671 DOI: 10.1542/peds.2016-2822] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/17/2017] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND AND OBJECTIVES In 2000, the US Congress authorized the National Institutes of Health to conduct a prospective national longitudinal study of environmental influences on children's health and development from birth through 21 years. Several recruitment methodologies were piloted to determine the optimal strategy for a main National Children's Study. METHODS After an initial pilot recruitment that used a household enumeration strategy performed poorly, the National Children's Study Vanguard Study developed and evaluated the feasibility, acceptability, and cost of 4 alternate strategies to recruit a large prospective national probability sample of pregnant women and their newborn children. We compare household-based recruitment, provider-based recruitment, direct outreach, and provider-based sampling (PBS) strategies with respect to overall recruitment success, efficiency, cost, and fulfillment of scientific requirements. RESULTS Although all 5 strategies achieved similar enrollment rates (63%-81%) among eligible women, PBS achieved the highest recruitment success as measured by the ratio of observed-to-expected newborn enrollees per year of 0.99, exceeding those of the other strategies (range: 0.35-0.48). Because PBS could reach the enrollment target through sampling of high volume obstetric provider offices and birth hospitals, it achieved the lowest ratio of women screened to women enrolled and was also the least costly strategy. With the exception of direct outreach, all strategies enrolled a cohort of women whose demographics were similar to county natality data. CONCLUSIONS PBS demonstrated the optimal combination of recruitment success, efficiency, cost, and population representativeness and serves as a model for the assembly of future prospective probability-based birth cohorts.
Collapse
Affiliation(s)
| | | | - Jennifer Kwan
- National Institute of Allergy and Infectious Diseases, Bethesda, Maryland
| | - Linda Andrews
- Social & Scientific Systems Inc, Silver Spring, Maryland; and
| | - Mark L. Hudak
- Department of Pediatrics, University of Florida College of Medicine – Jacksonville, Jacksonville, Florida
| |
Collapse
|
3
|
van Zon SKR, Scholtens S, Reijneveld SA, Smidt N, Bültmann U. Active recruitment and limited participant-load related to high participation in large population-based biobank studies. J Clin Epidemiol 2016; 78:52-62. [PMID: 27032874 DOI: 10.1016/j.jclinepi.2016.03.009] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2015] [Revised: 02/15/2016] [Accepted: 03/21/2016] [Indexed: 12/31/2022]
Abstract
OBJECTIVES Insight into baseline participation rates and their determinants is crucial for designing future population-based biobank studies. We therefore conducted a systematic review and meta-analysis of baseline participation rates and their determinants in large longitudinal population-based biobank studies. STUDY DESIGN AND SETTING We screened studies registered within the Public Population Project in Genomics and Society and in the Biobanking and Biomolecular Resources Research Infrastructure catalogues to find potentially eligible studies. We retrieved data with regard to participation rate, biobank design, performed measurements, and specific strategies for improving participation. We calculated weighted pooled proportions for each determinant using random-effects models. RESULTS We included 25 studies (participation rates 5-96%). Participation rates were highest for studies involving face-to-face recruitment [82.6%; 95% confidence interval (CI): 72.2%, 90.9%], for studies in which participants were visited for an examination (77.5%; 95% CI: 64.0%, 88.6%) and for studies in which at maximum four measurements were performed (78.2%; 95% CI: 69.7%, 85.7%). Specific strategies to improve participation were not found to be associated with higher participation rates. CONCLUSION Specific choices of recruitment methods and design have consequences for participation rates. These insights may help to increase participation in future studies, thereby enhancing the validity of their findings.
Collapse
Affiliation(s)
- Sander K R van Zon
- Department of Health Sciences, Community & Occupational Medicine, University Medical Center Groningen, University of Groningen, P.O Box 196, FA 10, Antonius Deusinglaan 1, 9700 AD Groningen, The Netherlands.
| | - Salome Scholtens
- LifeLines Cohort Study and Biobank, Bloemsingel 1, 9713 BZ Groningen, Groningen, The Netherlands
| | - Sijmen A Reijneveld
- Department of Health Sciences, Community & Occupational Medicine, University Medical Center Groningen, University of Groningen, P.O Box 196, FA 10, Antonius Deusinglaan 1, 9700 AD Groningen, The Netherlands
| | - Nynke Smidt
- Department of Epidemiology, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9713 GZ Groningen, Groningen, The Netherlands
| | - Ute Bültmann
- Department of Health Sciences, Community & Occupational Medicine, University Medical Center Groningen, University of Groningen, P.O Box 196, FA 10, Antonius Deusinglaan 1, 9700 AD Groningen, The Netherlands
| |
Collapse
|
4
|
Bray I, Noble S, Boyd A, Brown L, Hayes P, Malcolm J, Robinson R, Williams R, Burston K, Macleod J, Molloy L, Tilling K. A randomised controlled trial comparing opt-in and opt-out home visits for tracing lost participants in a prospective birth cohort study. BMC Med Res Methodol 2015. [PMID: 26202794 PMCID: PMC4512038 DOI: 10.1186/s12874-015-0041-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Attrition is an important problem in cohort studies. Tracing cohort members who have moved or otherwise lost contact with the study is vital. There is some debate about the acceptability and relative effectiveness of opt-in versus opt-out methods of contacting cohort members to re-engage them in this context. We conducted a randomised controlled trial to compare the two approaches in terms of effectiveness (tracing to confirm address and consenting to continue in the study), cost-effectiveness and acceptability. METHODS Participants in this trial were individuals (young people and mothers) recruited to the Avon Longitudinal Study of Parents and Children (ALSPAC), who had not engaged with the study in the previous 5 years and for whom mail had been returned from their last known address. The sampling frame was restricted to those for whom database searching led to a potential new address being found in the Bristol area. 300 participants were randomly selected and assigned using stratified randomisation to the opt-in or opt-out arm. A tailored letter was sent to the potential new address, either asking participants to opt in to a home visit, or giving them the option to opt out of a home visit. Fieldworkers from Ipsos MORI conducted home visits to confirm address details. RESULTS The proportion who were traced was higher in the opt-out arm (77/150 = 51 %) than the opt-in arm (6/150 = 4 %), as was the proportion who consented to continue in ALSPAC (46/150 = 31 % v 4/150 = 3 %). The mean cost per participant was £8.14 in the opt-in arm and £71.93 in the opt-out arm. There was no evidence of a difference in acceptability between the opt-in and opt-out approaches. CONCLUSION Since the opt-in approach yielded very low response rates, and there were no differences in terms of acceptability, we conclude that the opt-out approach is the most effective method of tracing disengaged study members. The gains made in contacting participants must be weighed against the increase in cost using this methodology.
Collapse
Affiliation(s)
- Isabelle Bray
- Department of Health and Social Science, University of the West of England, Frenchay Campus, Bristol, BS16 1QY, UK.
| | - Sian Noble
- School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
| | - Andy Boyd
- ALSPAC, School of Social and Community Medicine, University of Bristol, Oakfield House, Oakfield Grove, Bristol, BS8 2BN, UK.
| | - Lindsey Brown
- The Ethox Centre, Nuffield Department of Population Health, University of Oxford, Old Road Campus, Oxford, OX3 7LF, UK.
| | - Pei Hayes
- The Ethox Centre, Nuffield Department of Population Health, University of Oxford, Old Road Campus, Oxford, OX3 7LF, UK.
| | - Joanne Malcolm
- ALSPAC, School of Social and Community Medicine, University of Bristol, Oakfield House, Oakfield Grove, Bristol, BS8 2BN, UK.
| | - Ross Robinson
- ALSPAC, School of Social and Community Medicine, University of Bristol, Oakfield House, Oakfield Grove, Bristol, BS8 2BN, UK.
| | | | | | - John Macleod
- School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
| | - Lynn Molloy
- ALSPAC, School of Social and Community Medicine, University of Bristol, Oakfield House, Oakfield Grove, Bristol, BS8 2BN, UK.
| | - Kate Tilling
- School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
| |
Collapse
|
5
|
Vasan RS, Folsom AR. Invited commentary: future of population studies--defining research priorities and processes. Am J Epidemiol 2015; 181:369-71. [PMID: 25743322 DOI: 10.1093/aje/kwv012] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
In this issue of the Journal, an expert panel offers 7 recommendations on how population studies supported by National Heart, Lung, and Blood Institute contracts might be strategically transformed (Am J Epidemiol. 2015;181(6):363-368). The Institute and its external advisors seemingly established this panel of epidemiologists and nonepidemiologists primarily to find ways to save research costs. Although the working group's recommendations offer reasonable approaches, we believe that, even in tough fiscal times, the main drivers of cardiovascular epidemiologic research must remain 1) scientific questions that are important and 2) study designs to match these. Although cardiovascular epidemiology admittedly is often redundant and needs to be more efficient, undue focus on administrative efficiency and cost savings will not necessarily guarantee cutting-edge population research.
Collapse
|
6
|
McWhirter RE, Critchley CR, Nicol D, Chalmers D, Whitton T, Otlowski M, Burgess MM, Dickinson JL. Community engagement for big epidemiology: deliberative democracy as a tool. J Pers Med 2014; 4:459-74. [PMID: 25563457 PMCID: PMC4282883 DOI: 10.3390/jpm4040459] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2014] [Revised: 10/08/2014] [Accepted: 11/12/2014] [Indexed: 12/24/2022] Open
Abstract
Public trust is critical in any project requiring significant public support, both in monetary terms and to encourage participation. The research community has widely recognized the centrality of public trust, garnered through community consultation, to the success of large-scale epidemiology. This paper examines the potential utility of the deliberative democracy methodology within the public health research setting. A deliberative democracy event was undertaken in Tasmania, Australia, as part of a wider program of community consultation regarding the potential development of a Tasmanian Biobank. Twenty-five Tasmanians of diverse backgrounds participated in two weekends of deliberation; involving elements of information gathering; discussion; identification of issues and formation of group resolutions. Participants demonstrated strong support for a Tasmanian Biobank and their deliberations resulted in specific proposals in relation to consent; privacy; return of results; governance; funding; and, commercialization and benefit sharing. They exhibited a high degree of satisfaction with the event, and confidence in the outcomes. Deliberative democracy methodology is a useful tool for community engagement that addresses some of the limitations of traditional consultation methods.
Collapse
Affiliation(s)
- Rebekah E McWhirter
- Menzies Institute for Medical Research, University of Tasmania, Hobart TAS 7000, Australia.
| | - Christine R Critchley
- Centre for Law and Genetics, Faculty of Law, University of Tasmania, Hobart TAS 7001, Australia.
| | - Dianne Nicol
- Menzies Institute for Medical Research, University of Tasmania, Hobart TAS 7000, Australia.
| | - Don Chalmers
- Menzies Institute for Medical Research, University of Tasmania, Hobart TAS 7000, Australia.
| | - Tess Whitton
- Menzies Institute for Medical Research, University of Tasmania, Hobart TAS 7000, Australia.
| | - Margaret Otlowski
- Centre for Law and Genetics, Faculty of Law, University of Tasmania, Hobart TAS 7001, Australia.
| | - Michael M Burgess
- Maurice Young Centre for Applied Ethics, University of British Columbia, Vancouver, BC V6T 1Z2, Canada.
| | - Joanne L Dickinson
- Menzies Institute for Medical Research, University of Tasmania, Hobart TAS 7000, Australia.
| |
Collapse
|
7
|
Ogbogu U, Toews M, Ollenberger A, Borry P, Nobile H, Bergmann M, Caulfield T. Newspaper coverage of biobanks. PeerJ 2014; 2:e500. [PMID: 25101229 PMCID: PMC4121587 DOI: 10.7717/peerj.500] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2014] [Accepted: 07/11/2014] [Indexed: 12/22/2022] Open
Abstract
Background. Biobanks are an important research resource that provides researchers with biological samples, tools and data, but have also been associated with a range of ethical, legal and policy issues and concerns. Although there have been studies examining the views of different stakeholders, such as donors, researchers and the general public, the media portrayal of biobanks has been absent from this body of research. This study therefore examines how biobanking has been represented in major print newspapers from Australia, Canada, the United Kingdom and the United States to identify the issues and concerns surrounding biobanks that have featured most prominently in the print media discourse. Methods. Using Factiva, articles published in major broadsheet newspapers in Canada, the US, the UK, and Australia were identified using specified search terms. The final sample size consisted of 163 articles. Results. Majority of articles mentioned or discussed the benefits of biobanking, with medical research being the most prevalent benefit mentioned. Fewer articles discussed risks associated with biobanking. Researchers were the group of people most quoted in the articles, followed by biobank employees. Biobanking was portrayed as mostly neutral or positive, with few articles portraying biobanking in a negative manner. Conclusion. Reporting on biobanks in the print media heavily favours discussions of related benefits over risks. Members of the scientific research community appear to be a primary source of this positive tone. Under-reporting of risks and a downtrend in reporting on legal and regulatory issues suggests that the print media views such matters as less newsworthy than perceived benefits of biobanking.
Collapse
Affiliation(s)
- Ubaka Ogbogu
- Faculties of Law and Pharmacy & Pharmaceutical Sciences, University of Alberta , Edmonton, Alberta , Canada ; Health Law Institute, Faculty of Law, University of Alberta , Edmonton, Alberta , Canada
| | - Maeghan Toews
- Health Law Institute, Faculty of Law, University of Alberta , Edmonton, Alberta , Canada
| | - Adam Ollenberger
- Health Law Institute, Faculty of Law, University of Alberta , Edmonton, Alberta , Canada
| | - Pascal Borry
- Department of Public Health and Primary Care, KU Leuven , Leuven , Belgium
| | - Helene Nobile
- Department of Public Health and Primary Care, KU Leuven , Leuven , Belgium ; Department of Epidemiology, German Institute of Human Nutrition , Potsdam-Rehbruecke, Nuthetal , Germany
| | - Manuela Bergmann
- Department of Epidemiology, German Institute of Human Nutrition , Potsdam-Rehbruecke, Nuthetal , Germany
| | - Timothy Caulfield
- Health Law Institute, Faculty of Law, University of Alberta , Edmonton, Alberta , Canada ; Faculty of Law and School of Public Health, University of Alberta , Edmonton, Alberta , Canada
| |
Collapse
|
8
|
Abstract
PURPOSE OF REVIEW HIV-infected individuals are living longer as a result of effective treatment. Age-related comorbidities now account for the majority of morbidity and mortality among treated HIV-infected adults. Previous findings regarding the age at, and risk of, these comorbidities have been mixed, sparking debate in the field. Discerning potential differences in the occurrence and burden of age-related comorbidities among treated HIV-infected adults as compared with uninfected adults of the same age requires careful selection of the appropriate uninfected comparison group. RECENT FINDINGS The validity of comparisons with HIV-uninfected populations is threatened when differences in demographic, clinical, and lifestyle characteristics between HIV-infected and uninfected adults are not considered. Identifying a pool of HIV-uninfected individuals from existing secondary data resources and employing selection methodologies may be a novel approach to reduce threats to internal validity. Issues related to identifying data sources, understanding inclusion criteria, determining measurement error, and threats to inference are discussed. SUMMARY The development of clinical interventions targeting age-related comorbidities will rely on deriving valid inferences from appropriate comparison groups. The use of secondary data resources and selection methodology to create the appropriate uninfected comparison group is an attractive approach in the setting of finite resources, but are not without limitations.
Collapse
Affiliation(s)
- Cherise Wong
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Keri Althoff
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Stephen J. Gange
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| |
Collapse
|
9
|
Dal-Re R, Ioannidis JP, Bracken MB, Buffler PA, Chan AW, Franco EL, La Vecchia C, Weiderpass E. Making Prospective Registration of Observational Research a Reality. Sci Transl Med 2014; 6:224cm1. [DOI: 10.1126/scitranslmed.3007513] [Citation(s) in RCA: 87] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
10
|
Affiliation(s)
- Teri A Manolio
- Division of Genomic Medicine, National Human Genome Research Institute, 5635 Fishers Lane, Room 4113, MSC 9305, Rockville, MD 20892-9305, USA.
| | | |
Collapse
|