New models for large prospective studies: is there a risk of throwing out the baby with the bathwater?

Big data, observational research and P-value: a recipe for false-positive findings? A study of simulated and real prospective cohorts

BACKGROUND

An increasing number of observational studies combine large sample sizes with low participation rates, which could lead to standard inference failing to control the false-discovery rate. We investigated if the 'empirical calibration of P-value' method (EPCV), reliant on negative controls, can preserve type I error in the context of survival analysis.

METHODS

We used simulated cohort studies with 50% participation rate and two different selection bias mechanisms, and a real-life application on predictors of cancer mortality using data from four population-based cohorts in Northern Italy (n = 6976 men and women aged 25-74 years at baseline and 17 years of median follow-up).

RESULTS

Type I error for the standard Cox model was above the 5% nominal level in 15 out of 16 simulated settings; for n = 10 000, the chances of a null association with hazard ratio = 1.05 having a P-value < 0.05 were 42.5%. Conversely, EPCV with 10 negative controls preserved the 5% nominal level in all the simulation settings, reducing bias in the point estimate by 80-90% when its main assumption was verified. In the real case, 15 out of 21 (71%) blood markers with no association with cancer mortality according to literature had a P-value < 0.05 in age- and gender-adjusted Cox models. After calibration, only 1 (4.8%) remained statistically significant.

CONCLUSIONS

In the analyses of large observational studies prone to selection bias, the use of empirical distribution to calibrate P-values can substantially reduce the number of trivial results needing further screening for relevance and external validity.

Comparison of Recruitment Strategy Outcomes in the National Children's Study

BACKGROUND AND OBJECTIVES

In 2000, the US Congress authorized the National Institutes of Health to conduct a prospective national longitudinal study of environmental influences on children's health and development from birth through 21 years. Several recruitment methodologies were piloted to determine the optimal strategy for a main National Children's Study.

METHODS

After an initial pilot recruitment that used a household enumeration strategy performed poorly, the National Children's Study Vanguard Study developed and evaluated the feasibility, acceptability, and cost of 4 alternate strategies to recruit a large prospective national probability sample of pregnant women and their newborn children. We compare household-based recruitment, provider-based recruitment, direct outreach, and provider-based sampling (PBS) strategies with respect to overall recruitment success, efficiency, cost, and fulfillment of scientific requirements.

RESULTS

Although all 5 strategies achieved similar enrollment rates (63%-81%) among eligible women, PBS achieved the highest recruitment success as measured by the ratio of observed-to-expected newborn enrollees per year of 0.99, exceeding those of the other strategies (range: 0.35-0.48). Because PBS could reach the enrollment target through sampling of high volume obstetric provider offices and birth hospitals, it achieved the lowest ratio of women screened to women enrolled and was also the least costly strategy. With the exception of direct outreach, all strategies enrolled a cohort of women whose demographics were similar to county natality data.

CONCLUSIONS

PBS demonstrated the optimal combination of recruitment success, efficiency, cost, and population representativeness and serves as a model for the assembly of future prospective probability-based birth cohorts.

Active recruitment and limited participant-load related to high participation in large population-based biobank studies

OBJECTIVES

Insight into baseline participation rates and their determinants is crucial for designing future population-based biobank studies. We therefore conducted a systematic review and meta-analysis of baseline participation rates and their determinants in large longitudinal population-based biobank studies.

STUDY DESIGN AND SETTING

We screened studies registered within the Public Population Project in Genomics and Society and in the Biobanking and Biomolecular Resources Research Infrastructure catalogues to find potentially eligible studies. We retrieved data with regard to participation rate, biobank design, performed measurements, and specific strategies for improving participation. We calculated weighted pooled proportions for each determinant using random-effects models.

RESULTS

We included 25 studies (participation rates 5-96%). Participation rates were highest for studies involving face-to-face recruitment [82.6%; 95% confidence interval (CI): 72.2%, 90.9%], for studies in which participants were visited for an examination (77.5%; 95% CI: 64.0%, 88.6%) and for studies in which at maximum four measurements were performed (78.2%; 95% CI: 69.7%, 85.7%). Specific strategies to improve participation were not found to be associated with higher participation rates.

CONCLUSION

Specific choices of recruitment methods and design have consequences for participation rates. These insights may help to increase participation in future studies, thereby enhancing the validity of their findings.

A randomised controlled trial comparing opt-in and opt-out home visits for tracing lost participants in a prospective birth cohort study

BACKGROUND

Attrition is an important problem in cohort studies. Tracing cohort members who have moved or otherwise lost contact with the study is vital. There is some debate about the acceptability and relative effectiveness of opt-in versus opt-out methods of contacting cohort members to re-engage them in this context. We conducted a randomised controlled trial to compare the two approaches in terms of effectiveness (tracing to confirm address and consenting to continue in the study), cost-effectiveness and acceptability.

METHODS

Participants in this trial were individuals (young people and mothers) recruited to the Avon Longitudinal Study of Parents and Children (ALSPAC), who had not engaged with the study in the previous 5 years and for whom mail had been returned from their last known address. The sampling frame was restricted to those for whom database searching led to a potential new address being found in the Bristol area. 300 participants were randomly selected and assigned using stratified randomisation to the opt-in or opt-out arm. A tailored letter was sent to the potential new address, either asking participants to opt in to a home visit, or giving them the option to opt out of a home visit. Fieldworkers from Ipsos MORI conducted home visits to confirm address details.

RESULTS

The proportion who were traced was higher in the opt-out arm (77/150 = 51 %) than the opt-in arm (6/150 = 4 %), as was the proportion who consented to continue in ALSPAC (46/150 = 31 % v 4/150 = 3 %). The mean cost per participant was £8.14 in the opt-in arm and £71.93 in the opt-out arm. There was no evidence of a difference in acceptability between the opt-in and opt-out approaches.

CONCLUSION

Since the opt-in approach yielded very low response rates, and there were no differences in terms of acceptability, we conclude that the opt-out approach is the most effective method of tracing disengaged study members. The gains made in contacting participants must be weighed against the increase in cost using this methodology.

Invited commentary: future of population studies--defining research priorities and processes

In this issue of the Journal, an expert panel offers 7 recommendations on how population studies supported by National Heart, Lung, and Blood Institute contracts might be strategically transformed (Am J Epidemiol. 2015;181(6):363-368). The Institute and its external advisors seemingly established this panel of epidemiologists and nonepidemiologists primarily to find ways to save research costs. Although the working group's recommendations offer reasonable approaches, we believe that, even in tough fiscal times, the main drivers of cardiovascular epidemiologic research must remain 1) scientific questions that are important and 2) study designs to match these. Although cardiovascular epidemiology admittedly is often redundant and needs to be more efficient, undue focus on administrative efficiency and cost savings will not necessarily guarantee cutting-edge population research.

Community engagement for big epidemiology: deliberative democracy as a tool

Public trust is critical in any project requiring significant public support, both in monetary terms and to encourage participation. The research community has widely recognized the centrality of public trust, garnered through community consultation, to the success of large-scale epidemiology. This paper examines the potential utility of the deliberative democracy methodology within the public health research setting. A deliberative democracy event was undertaken in Tasmania, Australia, as part of a wider program of community consultation regarding the potential development of a Tasmanian Biobank. Twenty-five Tasmanians of diverse backgrounds participated in two weekends of deliberation; involving elements of information gathering; discussion; identification of issues and formation of group resolutions. Participants demonstrated strong support for a Tasmanian Biobank and their deliberations resulted in specific proposals in relation to consent; privacy; return of results; governance; funding; and, commercialization and benefit sharing. They exhibited a high degree of satisfaction with the event, and confidence in the outcomes. Deliberative democracy methodology is a useful tool for community engagement that addresses some of the limitations of traditional consultation methods.

Newspaper coverage of biobanks

Background. Biobanks are an important research resource that provides researchers with biological samples, tools and data, but have also been associated with a range of ethical, legal and policy issues and concerns. Although there have been studies examining the views of different stakeholders, such as donors, researchers and the general public, the media portrayal of biobanks has been absent from this body of research. This study therefore examines how biobanking has been represented in major print newspapers from Australia, Canada, the United Kingdom and the United States to identify the issues and concerns surrounding biobanks that have featured most prominently in the print media discourse. Methods. Using Factiva, articles published in major broadsheet newspapers in Canada, the US, the UK, and Australia were identified using specified search terms. The final sample size consisted of 163 articles. Results. Majority of articles mentioned or discussed the benefits of biobanking, with medical research being the most prevalent benefit mentioned. Fewer articles discussed risks associated with biobanking. Researchers were the group of people most quoted in the articles, followed by biobank employees. Biobanking was portrayed as mostly neutral or positive, with few articles portraying biobanking in a negative manner. Conclusion. Reporting on biobanks in the print media heavily favours discussions of related benefits over risks. Members of the scientific research community appear to be a primary source of this positive tone. Under-reporting of risks and a downtrend in reporting on legal and regulatory issues suggests that the print media views such matters as less newsworthy than perceived benefits of biobanking.

Identifying the appropriate comparison group for HIV-infected individuals

PURPOSE OF REVIEW

HIV-infected individuals are living longer as a result of effective treatment. Age-related comorbidities now account for the majority of morbidity and mortality among treated HIV-infected adults. Previous findings regarding the age at, and risk of, these comorbidities have been mixed, sparking debate in the field. Discerning potential differences in the occurrence and burden of age-related comorbidities among treated HIV-infected adults as compared with uninfected adults of the same age requires careful selection of the appropriate uninfected comparison group.

RECENT FINDINGS

The validity of comparisons with HIV-uninfected populations is threatened when differences in demographic, clinical, and lifestyle characteristics between HIV-infected and uninfected adults are not considered. Identifying a pool of HIV-uninfected individuals from existing secondary data resources and employing selection methodologies may be a novel approach to reduce threats to internal validity. Issues related to identifying data sources, understanding inclusion criteria, determining measurement error, and threats to inference are discussed.

SUMMARY

The development of clinical interventions targeting age-related comorbidities will rely on deriving valid inferences from appropriate comparison groups. The use of secondary data resources and selection methodology to create the appropriate uninfected comparison group is an attractive approach in the setting of finite resources, but are not without limitations.