Excess success for three related papers on racial bias

Reporting in Experimental Philosophy: Current Standards and Recommendations for Future Practice

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods (n = 134) that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact (numbers of citations). We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible.

Race as Social Construct

It is often claimed that race is a social construct and that scientists studying race differences are disruptive racists. The recent April 2018 “Race Issue” of the widely distributed National Geographic Magazine (NG) provided its millions of readers with a particularly illustrative example of this position. As discussions of race issues often recur, in both scientific and lay literature, stir considerable polemics, and have political, societal and human implications, we found it of both scientific and general interest to identify and dissect the following partly overlapping key contentions of the NG race issue magazine: (1) Samuel Morton’s studies of brain size is reprehensible racism (2) Race does not relate to geographic location, (3) Races do not exist as we are all equals and Africans, (4) Admixture and displacement erase race differences as soon as they appear, and (5) Race is only skin color deep. Also examined is the claim that Race does not matter. When analyzed within syllogistic formalism, each of the claims is found theoretically and empirically unsustainable, as Morton’s continuously evolving race position is misrepresented, race relates significantly to geography, we are far from equals, races have definitely not been erased, and race, whether self-reported or defined by ancestry, lineage, ecotype, species, or genes, is much more than skin color deep. Race matters vitally for people and societies. We conclude that important research on existing population differences is hurt when widely respected institutions such as NG mobilize their full authority in a massively circulated attempt to betray its scientific and public readership by systematically misrepresenting historical sources and scientific positions, shaming past scientists, and by selectively suppressing unwanted or unacceptable results–acts included as examples of academic fraud by the National Academy of Sciences (US, 1986). Any unqualified a priori denial of the formative evolutionary aspects of individual and population differences threatens to impede the recent promising research on effects of genome wide allelic associations, which would lames us in the vital quest to develop rational solutions to associated globally pressing societal problems.

Implications of "Too Good to Be True" for Replication, Theoretical Claims, and Experimental Design: An Example Using Prominent Studies of Racial Bias

In response to concerns about the validity of empirical findings in psychology, some scientists use replication studies as a way to validate good science and to identify poor science. Such efforts are resource intensive and are sometimes controversial (with accusations of researcher incompetence) when a replication fails to show a previous result. An alternative approach is to examine the statistical properties of the reported literature to identify some cases of poor science. This review discusses some details of this process for prominent findings about racial bias, where a set of studies seems “too good to be true.” This kind of analysis is based on the original studies, so it avoids criticism from the original authors about the validity of replication studies. The analysis is also much easier to perform than a new empirical study. A variation of the analysis can also be used to explore whether it makes sense to run a replication study. As demonstrated here, there are situations where the existing data suggest that a direct replication of a set of studies is not worth the effort. Such a conclusion should motivate scientists to generate alternative experimental designs that better test theoretical ideas.