Consensus-based guidance for conducting and reporting multi-analyst studies

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Putting the Self in Self-Correction: Findings From the Loss-of-Confidence Project

Science is often perceived to be a self-correcting enterprise. In principle, the assessment of scientific claims is supposed to proceed in a cumulative fashion, with the reigning theories of the day progressively approximating truth more accurately over time. In practice, however, cumulative self-correction tends to proceed less efficiently than one might naively suppose. Far from evaluating new evidence dispassionately and infallibly, individual scientists often cling stubbornly to prior findings. Here we explore the dynamics of scientific self-correction at an individual rather than collective level. In 13 written statements, researchers from diverse branches of psychology share why and how they have lost confidence in one of their own published findings. We qualitatively characterize these disclosures and explore their implications. A cross-disciplinary survey suggests that such loss-of-confidence sentiments are surprisingly common among members of the broader scientific population yet rarely become part of the public record. We argue that removing barriers to self-correction at the individual level is imperative if the scientific community as a whole is to achieve the ideal of efficient self-correction.

Scientific Utopia III: Crowdsourcing Science

Most scientific research is conducted by small teams of investigators who together formulate hypotheses, collect data, conduct analyses, and report novel findings. These teams operate independently as vertically integrated silos. Here we argue that scientific research that is horizontally distributed can provide substantial complementary value, aiming to maximize available resources, promote inclusiveness and transparency, and increase rigor and reliability. This alternative approach enables researchers to tackle ambitious projects that would not be possible under the standard model. Crowdsourced scientific initiatives vary in the degree of communication between project members from largely independent work curated by a coordination team to crowd collaboration on shared activities. The potential benefits and challenges of large-scale collaboration span the entire research process: ideation, study design, data collection, data analysis, reporting, and peer review. Complementing traditional small science with crowdsourced approaches can accelerate the progress of science and improve the quality of scientific research.

Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 ( Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy in which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective, analytic choices influence research results.

Managerial Coordination Challenges in the Alignment of Capabilities and New Subsidiary Charters in MNEs

Subsidiary-level change requires the alignment of subsidiary charters and capabilities. Yet, the mechanisms through which the alignment of charters and capabilities unfolds are not yet well understood. In this paper, we investigate alignment from the perspective of managerial coordination. Drawing on a longitudinal study of a global IT firm, we identify three coordination mechanisms (charter-, experience-, and interaction-based coordination). By tracing the shifts in these coordination mechanisms over time and by specifying the implications of each mechanism for capability level change, we explain how managerial coordination influences alignment via subsidiary level capability change as well as alignment via the potential renegotiation of charters. This also allows us to provide new insights into situations of misalignment by explaining that particular mechanisms of coordination may become a source of decoupling between subsidiary actions and HQ mandates and may also result in capability level inertia. Moreover, while prior research has already acknowledged the role of interaction-based coordination for capability level change we show how and why such a mechanism of coordination emerges.

It Pays to Be Herr Kaiser

In the field study reported here ( N = 222,924), we found that Germans with noble-sounding surnames, such as Kaiser (“emperor”), König (“king”), and Fürst (“prince”), more frequently hold managerial positions than Germans with last names that either refer to common everyday occupations, such as Koch (“cook”), Bauer (“farmer”), and Becker/Bäcker (“baker”), or do not refer to any social role. This phenomenon occurs despite the fact that noble-sounding surnames never indicated that the person actually held a noble title. Because of basic properties of associative cognition, the status linked to a name may spill over to its bearer and influence his or her occupational outcomes.