Bayesian inference under cluster sampling with probability proportional to size

Cluster sampling is common in survey practice, and the corresponding inference has been predominantly design based. We develop a Bayesian framework for cluster sampling and account for the design effect in the outcome modeling. We consider a two-stage cluster sampling design where the clusters are first selected with probability proportional to cluster size, and then units are randomly sampled inside selected clusters. Challenges arise when the sizes of the nonsampled cluster are unknown. We propose nonparametric and parametric Bayesian approaches for predicting the unknown cluster sizes, with this inference performed simultaneously with the model for survey outcome, with computation performed in the open-source Bayesian inference engine Stan. Simulation studies show that the integrated Bayesian approach outperforms classical methods with efficiency gains, especially under informative cluster sampling design with small number of selected clusters. We apply the method to the Fragile Families and Child Wellbeing study as an illustration of inference for complex health surveys.

Do Researchers Anchor Their Beliefs on the Outcome of an Initial Study?

As a research field expands, scientists have to update their knowledge and integrate the outcomes of a sequence of studies. However, such integrative judgments are generally known to fall victim to a primacy bias where people anchor their judgments on the initial information. In this preregistered study we tested the hypothesis that people anchor on the outcome of a small initial study, reducing the impact of a larger subsequent study that contradicts the initial result. Contrary to our expectation, undergraduates and academics displayed a recency bias, anchoring their judgment on the research outcome presented last. This recency bias is due to the fact that unsuccessful replications decreased trust in an effect more than did unsuccessful initial experiments. We recommend the time-reversal heuristic to account for temporal order effects during integration of research results.

Global shifts in the phenological synchrony of species interactions over recent decades

Phenological responses to climate change (e.g., earlier leaf-out or egg hatch date) are now well documented and clearly linked to rising temperatures in recent decades. Such shifts in the phenologies of interacting species may lead to shifts in their synchrony, with cascading community and ecosystem consequences. To date, single-system studies have provided no clear picture, either finding synchrony shifts may be extremely prevalent [Mayor SJ, et al. (2017) Sci Rep 7:1902] or relatively uncommon [Iler AM, et al. (2013) Glob Chang Biol 19:2348-2359], suggesting that shifts toward asynchrony may be infrequent. A meta-analytic approach would provide insights into global trends and how they are linked to climate change. We compared phenological shifts among pairwise species interactions (e.g., predator-prey) using published long-term time-series data of phenological events from aquatic and terrestrial ecosystems across four continents since 1951 to determine whether recent climate change has led to overall shifts in synchrony. We show that the relative timing of key life cycle events of interacting species has changed significantly over the past 35 years. Further, by comparing the period before major climate change (pre-1980s) and after, we show that estimated changes in phenology and synchrony are greater in recent decades. However, there has been no consistent trend in the direction of these changes. Our findings show that there have been shifts in the timing of interacting species in recent decades; the next challenges are to improve our ability to predict the direction of change and understand the full consequences for communities and ecosystems.

The Millennium Villages Project: a retrospective, observational, endline evaluation

BACKGROUND

The Millennium Villages Project (MVP) was a 10 year, multisector, rural development project, initiated in 2005, operating across ten sites in ten sub-Saharan African countries to achieve the Millennium Development Goals (MDGs). In this study, we aimed to estimate the project's impact, target attainment, and on-site spending.

METHODS

In this endline evaluation of the MVP, we retrospectively selected comparison villages that best matched the project villages on possible confounding variables. Cross-sectional survey data on 40 outcomes of interest were collected from both the project and the comparison villages in 2015. Using these data, as well as on-site spending data collected during the project, we estimated project impacts as differences in outcomes between the project and comparison villages; target attainment as differences between project outcomes and prespecified targets; and on-site spending as expenditures reported by communities, donors, governments, and the project. Spending data were not collected in the comparison villages.

FINDINGS

Averaged across the ten project sites, we found that impact estimates for 30 of 40 outcomes were significant (95% uncertainty intervals [UIs] for these outcomes excluded zero) and favoured the project villages. In particular, substantial effects were seen in agriculture and health, in which some outcomes were roughly one SD better in the project villages than in the comparison villages. The project was estimated to have no significant impact on the consumption-based measures of poverty, but a significant favourable impact on an index of asset ownership. Impacts on nutrition and education outcomes were often inconclusive (95% UIs included zero). Averaging across outcomes within categories, the project had significant favourable impacts on agriculture, nutrition, education, child health, maternal health, HIV and malaria, and water and sanitation. A third of the targets were met in the project sites. Total on-site spending decreased from US$132 per person in the first half of the project (of which $66 was from the MVP) to $109 per person in the second half of the project (of which $25 was from the MVP).

INTERPRETATION

The MVP had favourable impacts on outcomes in all MDG areas, consistent with an integrated rural development approach. The greatest effects were in agriculture and health, suggesting support for the project's emphasis on agriculture and health systems strengthening. The project conclusively met one third of its targets.

FUNDING

The Open Society Foundations, the Islamic Development Bank, and the governments of Japan, South Korea, Mali, Senegal, and Uganda.

Exploring the Relationships Between USMLE Performance and Disciplinary Action in Practice: A Validity Study of Score Inferences From a Licensure Examination

PURPOSE

Physicians must pass the United States Medical Licensing Examination (USMLE) to obtain an unrestricted license to practice allopathic medicine in the United States. Little is known, however, about how well USMLE performance relates to physician behavior in practice, particularly conduct inconsistent with safe, effective patient care. The authors examined the extent to which USMLE scores relate to the odds of receiving a disciplinary action from a U.S. state medical board.

METHOD

Controlling for multiple factors, the authors used non-nested multilevel logistic regression analyses to estimate the relationships between scores and receiving an action. The sample included 164,725 physicians who graduated from U.S. MD-granting medical schools between 1994 and 2006.

RESULTS

Physicians had a mean Step 1 score of 214 (standard deviation [SD] = 21) and a mean Step 2 Clinical Knowledge (CK) score of 213 (SD = 23). Of the physicians, 2,205 (1.3%) received at least one action. Physicians with higher Step 2 CK scores had lower odds of receiving an action. A 1-SD increase in Step 2 CK scores corresponded to a decrease in the chance of disciplinary action by roughly 25% (odds ratio = 0.75; 95% CI = 0.70-0.80). After accounting for Step 2 CK scores, Step 1 scores were unrelated to the odds of receiving an action.

CONCLUSIONS

USMLE Step 2 CK scores provide useful information about the odds a physician will receive an official sanction for problematic practice behavior. These results provide validity evidence supporting current interpretation and use of Step 2 CK scores.

Increasing Transparency Through a Multiverse Analysis

Empirical research inevitably includes constructing a data set by processing raw data into a form ready for statistical analysis. Data processing often involves choices among several reasonable options for excluding, transforming, and coding data. We suggest that instead of performing only one analysis, researchers could perform a multiverse analysis, which involves performing all analyses across the whole set of alternatively processed data sets corresponding to a large set of reasonable scenarios. Using an example focusing on the effect of fertility on religiosity and political attitudes, we show that analyzing a single data set can be misleading and propose a multiverse analysis as an alternative practice. A multiverse analysis offers an idea of how much the conclusions change because of arbitrary choices in data construction and gives pointers as to which choices are most consequential in the fragility of the result.

The Failure of Null Hypothesis Significance Testing When Studying Incremental Changes, and What to Do About It

A standard mode of inference in social and behavioral science is to establish stylized facts using statistical significance in quantitative studies. However, in a world in which measurements are noisy and effects are small, this will not work: selection on statistical significance leads to effect sizes which are overestimated and often in the wrong direction. After a brief discussion of two examples, one in economics and one in social psychology, we consider the procedural solution of open postpublication review, the design solution of devoting more effort to accurate measurements and within-person comparisons, and the statistical analysis solution of multilevel modeling and reporting all results rather than selection on significance. We argue that the current replication crisis in science arises in part from the ill effects of null hypothesis significance testing being used to study small effects with noisy data. In such settings, apparent success comes easy but truly replicable results require a more serious connection between theory, measurement, and data.

A child with resistant Kawasaki disease successfully treated with anakinra: a case report

Background

Kawasaki disease (KD) is an acute self-limited systemic vasculitis of unknown etiology. Intravenous immunoglobulin (IVIG) is an effective treatment and decreases the risk of cardiac complications to less than 5%. In spite of its effectiveness, some children do not respond to this therapy and still develop coronary aneurysms (CAA). The optimal treatment for IVIG non-responsive patients remains controversial although corticoids have been suggested to be an effective treatment in some patients. For those patients still resistant to IVIG and corticoids, interleukin-1 receptor antagonists (IL-1RA) such anakinra could be an alternative.

Case presentation

We present a 3 year-old Caucasian patient with KD without cardiac complications but with important resistance to treatment. After becoming resistant to IVIG and corticoids, anakinra proved to be an effective treatment.

Conclusions

To our knowledge, this is the first report of the utility of IL-1RA in refractory KD without coronary impairment. The patient fulfilled the classical criteria for KD and, after becoming resistant to first and second line treatments, anakinra proved to be an effective treatment. Further studies are required to determine if this is an effective treatment option for other cases of resistant Kawasaki disease.