Raising the bar for reproducible science at the U.S. Environmental Protection Agency Office of Research and Development

Ethylene Oxide: An Air Contaminant of Concern

Ethylene oxide (EtO) is a colorless, flammable, reactive gas commonly used for sterilization and chemical manufacturing. It has become a contaminant of concern for the United States Environmental Protection Agency (USEPA) due to an assessment of its toxicity, which found that EtO is more potent than had been previously understood and which also revised the weight-of-evidence classification of EtO from "probably carcinogenic" to "carcinogenic to humans". With the revised toxicity assessment came findings of increased cancer risk to communities near some facilities that emit EtO to ambient air, including communities with environmental justice (EJ) concerns. To address EtO, the USEPA has conducted intensive research in recent years, centering its attention on measurement and sampling technology development, as well as monitoring of EtO in source emissions, near-source air, and atmospheric environments to further support science-based policy and regulations that reduce harmful impacts to human health. Research efforts by government, academic, and commercial institutions have resulted in the development of novel measurement and monitoring techniques, which has led to more robust characterization of EtO emissions and atmospheric levels across a wide range of concentrations, including trace levels (ppt). This Perspective covers the importance of capturing high quality, analytical measurements of EtO, what is known so far about these measurement technologies, EPA's response to the increasing concerns of EtO contamination, what still needs to be accomplished on the air quality front, and a focus on USEPA research and development moving forward.

Censoring Trace-Level Environmental Data: Statistical Analysis Considerations to Limit Bias

Trace-level environmental data typically include values near or below detection and quantitation thresholds where health effects may result from low-concentration exposures to one chemical over time or to multiple chemicals. In a cook stove case study, bias in dibenzo[a,h]anthracene concentration means and standard deviations (SDs) was assessed following censoring at thresholds for selected analysis approaches: substituting threshold/2, maximum likelihood estimation, robust regression on order statistics, Kaplan-Meier, and omitting censored observations. Means and SDs for gas chromatography-mass spectrometry-determined concentrations were calculated after censoring at detection and calibration thresholds, 17% and 55% of the data, respectively. Threshold/2 substitution was the least biased. Measurement values were subsequently simulated from two log-normal distributions at two sample sizes. Means and SDs were calculated for 30%, 50%, and 80% censoring levels and compared to known distribution counterparts. Simulation results illustrated (1) threshold/2 substitution to be inferior to modern after-censoring statistical approaches and (2) all after-censoring approaches to be inferior to including all measurement data in analysis. Additionally, differences in stove-specific group means were tested for uncensored samples and after censoring. Group differences of means tests varied depending on censoring and distributional decisions. Investigators should guard against censoring-related bias from (explicit or implicit) distributional and analysis approach decisions.

The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing

In the last few years, stakeholders in the scientific community have raised alarms about a perceived lack of reproducibility of scientific results. In reaction, guidelines for journals have been promulgated and grant applicants have been asked to address the rigor and reproducibility of their proposed projects. Neither solution addresses a primary culprit, which is the culture of null hypothesis significance testing that dominates statistical analysis and inference. In an innovative research enterprise, selection of results for further evaluation based on null hypothesis significance testing is doomed to yield a low proportion of reproducible results and a high proportion of effects that are initially overestimated. In addition, the culture of null hypothesis significance testing discourages quantitative adjustments to account for systematic errors and quantitative incorporation of prior information. These strategies would otherwise improve reproducibility and have not been previously proposed in the widely cited literature on this topic. Without discarding the culture of null hypothesis significance testing and implementing these alternative methods for statistical analysis and inference, all other strategies for improving reproducibility will yield marginal gains at best.

An academic researcher's guide to increased impact on regulatory assessment of chemicals

The interactions between academic research and regulatory assessment of chemicals may in theory seem straightforward: researchers perform studies, and these studies are used by regulators for decision-making. However, in practice the situation is more complex, and many factors decide a research study's regulatory use. According to several EU chemical legislations, all available and relevant studies can be used in hazard and risk assessment of chemicals. However, in practice, standard tests conducted under GLP and sponsored and provided by industry are predominantly used. Peer-reviewed studies from independent sources are often disregarded or disputed since they often do not comply with regulatory data requirements and quality criteria. To help bridge such a gap, the aim of this paper is to give an overview of the general workings of legislation of chemicals and propose a set of actions to increase the usability of research data. In the end, this may increase the use of academic research for decision-making and ultimately result in more science-based policies. From a policy perspective, useful scientific evidence comprises those studies that are sufficiently reliable and relevant. This is not in contradiction to the aims of research and generally accepted scientific standards.

The influence of control group reproduction on the statistical power of the Environmental Protection Agency's Medaka Extended One Generation Reproduction Test (MEOGRT)

Because of various Congressional mandates to protect the environment from endocrine disrupting chemicals (EDCs), the United States Environmental Protection Agency (USEPA) initiated the Endocrine Disruptor Screening Program. In the context of this framework, the Office of Research and Development within the USEPA developed the Medaka Extended One Generation Reproduction Test (MEOGRT) to characterize the endocrine action of a suspected EDC. One important endpoint of the MEOGRT is fecundity of medaka breeding pairs. Power analyses were conducted to determine the number of replicates needed in proposed test designs and to determine the effects that varying reproductive parameters (e.g. mean fecundity, variance, and days with no egg production) would have on the statistical power of the test. The MEOGRT Reproduction Power Analysis Tool (MRPAT) is a software tool developed to expedite these power analyses by both calculating estimates of the needed reproductive parameters (e.g. population mean and variance) and performing the power analysis under user specified scenarios. Example scenarios are detailed that highlight the importance of the reproductive parameters on statistical power. When control fecundity is increased from 21 to 38 eggs per pair per day and the variance decreased from 49 to 20, the gain in power is equivalent to increasing replication by 2.5 times. On the other hand, if 10% of the breeding pairs, including controls, do not spawn, the power to detect a 40% decrease in fecundity drops to 0.54 from nearly 0.98 when all pairs have some level of egg production. Perhaps most importantly, MRPAT was used to inform the decision making process that lead to the final recommendation of the MEOGRT to have 24 control breeding pairs and 12 breeding pairs in each exposure group.

Advanced Research and Data Methods in Women's Health: Big Data Analytics, Adaptive Studies, and the Road Ahead

Technical advances in science have had broad implications in reproductive and women's health care. Recent innovations in population-level data collection and storage have made available an unprecedented amount of data for analysis while computational technology has evolved to permit processing of data previously thought too dense to study. "Big data" is a term used to describe data that are a combination of dramatically greater volume, complexity, and scale. The number of variables in typical big data research can readily be in the thousands, challenging the limits of traditional research methodologies. Regardless of what it is called, advanced data methods, predictive analytics, or big data, this unprecedented revolution in scientific exploration has the potential to dramatically assist research in obstetrics and gynecology broadly across subject matter. Before implementation of big data research methodologies, however, potential researchers and reviewers should be aware of strengths, strategies, study design methods, and potential pitfalls. Examination of big data research examples contained in this article provides insight into the potential and the limitations of this data science revolution and practical pathways for its useful implementation.

Scientific integrity resource guide: Efforts by federal agencies, foundations, nonprofit organizations, professional societies, and academia in the United States

Scientific integrity is at the forefront of the scientific research enterprise. This paper provides an overview of key existing efforts on scientific integrity by federal agencies, foundations, nonprofit organizations, professional societies, and academia from 1989 to April 2016. It serves as a resource for the scientific community on scientific integrity work and helps to identify areas in which more action is needed. Overall, there is tremendous activity in this area and there are clear linkages among the efforts of the five sectors. All the same, scientific integrity needs to remain visible in the scientific community and evolve along with new research paradigms. High priority in instilling these values falls upon all stakeholders.

Using Inverse Probability Bootstrap Sampling to Eliminate Sample Induced Bias in Model Based Analysis of Unequal Probability Samples

In ecology, as in other research fields, efficient sampling for population estimation often drives sample designs toward unequal probability sampling, such as in stratified sampling. Design based statistical analysis tools are appropriate for seamless integration of sample design into the statistical analysis. However, it is also common and necessary, after a sampling design has been implemented, to use datasets to address questions that, in many cases, were not considered during the sampling design phase. Questions may arise requiring the use of model based statistical tools such as multiple regression, quantile regression, or regression tree analysis. However, such model based tools may require, for ensuring unbiased estimation, data from simple random samples, which can be problematic when analyzing data from unequal probability designs. Despite numerous method specific tools available to properly account for sampling design, too often in the analysis of ecological data, sample design is ignored and consequences are not properly considered. We demonstrate here that violation of this assumption can lead to biased parameter estimates in ecological research. In addition, to the set of tools available for researchers to properly account for sampling design in model based analysis, we introduce inverse probability bootstrapping (IPB). Inverse probability bootstrapping is an easily implemented method for obtaining equal probability re-samples from a probability sample, from which unbiased model based estimates can be made. We demonstrate the potential for bias in model-based analyses that ignore sample inclusion probabilities, and the effectiveness of IPB sampling in eliminating this bias, using both simulated and actual ecological data. For illustration, we considered three model based analysis tools--linear regression, quantile regression, and boosted regression tree analysis. In all models, using both simulated and actual ecological data, we found inferences to be biased, sometimes severely, when sample inclusion probabilities were ignored, while IPB sampling effectively produced unbiased parameter estimates.